-- Logs begin at Thu 2022-07-28 08:29:10 UTC, end at Thu 2022-07-28 09:36:46 UTC. -- Jul 28 08:56:45.656086 dv-cinder-851249-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 08:56:45.664216 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 08:56:45 2022] *** Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: nodename: dv-cinder-851249-5-iscsi Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: machine: x86_64 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: clock source: unix Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: pcre jit disabled Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: detected number of CPU cores: 4 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: current working directory: / Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: detected binary path: /usr/bin/uwsgi-core Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: your processes number limit is 47680 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: your memory page size is 4096 bytes Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: detected max file descriptor number: 2048 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: lock engine: pthread robust mutexes Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: thunder lock: enabled Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 08:56:45.668776 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 08:56:45.707598 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: Python main interpreter initialized at 0x564aadc3b9c0 Jul 28 08:56:45.707598 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: python threads support enabled Jul 28 08:56:45.707598 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: your server socket listen backlog is limited to 100 connections Jul 28 08:56:45.707598 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: your mercy for graceful operations on workers is 90 seconds Jul 28 08:56:45.707988 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 08:56:45.708056 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: *** Operational MODE: preforking *** Jul 28 08:56:45.708331 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: *** uWSGI is running in multiple interpreter mode *** Jul 28 08:56:45.708331 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: spawned uWSGI master process (pid: 80366) Jul 28 08:56:45.708531 dv-cinder-851249-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 08:56:45.709096 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: spawned uWSGI worker 1 (pid: 80374, cores: 1) Jul 28 08:56:45.709422 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: spawned uWSGI worker 2 (pid: 80375, cores: 1) Jul 28 08:56:45.710371 dv-cinder-851249-5-iscsi devstack@keystone.service[80366]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 08:56:51.246433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:56:51.246433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: from cryptography.utils import int_from_bytes Jul 28 08:56:51.247259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:56:51.247259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: from cryptography.utils import int_from_bytes Jul 28 08:56:51.272272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:56:51.272272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: from cryptography.utils import int_from_bytes Jul 28 08:56:51.272885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 08:56:51.272885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: from cryptography.utils import int_from_bytes Jul 28 08:56:51.298008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:56:51.298213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:56:51.298383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] command line args: [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:56:51.298656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:56:51.298833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ================================================================================ {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:56:51.299085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] admin_token = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.299332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] config_dir = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.299520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.299702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] config_source = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.299881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] debug = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.300082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.300594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.300691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.300884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.301134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.301366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.301546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_config_append = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.301725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.301988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_dir = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.302114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_file = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.302358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.302591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.302840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.303091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.303270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.303448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.303695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.303943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.304444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.304678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.304915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.305150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.305400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.305648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.305953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.306489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.306671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] publish_errors = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.306907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.307141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.307320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.307569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.307805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.308043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] standard_threads = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.308274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.308519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.308696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.308872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] use_journal = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.309049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] use_json = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.309224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] use_stderr = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.309399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] use_syslog = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.309632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.309925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.310168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.310437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.310677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.310927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.311167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.311417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.311655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.311908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.312225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.312527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.312766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.313048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.313287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.313539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.313790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.314061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.314318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.314573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.314821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.315061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.315317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.315584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.315838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.316089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.316322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.316555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.316788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.317023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.317276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.317526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.317757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.318015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.318266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.318515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.318755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.319006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.319237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.319474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.319710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.319941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.320173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.320409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.320642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.320879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.321131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.321363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.321602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.321833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.322096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.322332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.322579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.322810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.323049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.323286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.323522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.323741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 08:56:51.323831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.323951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 08:56:51.324035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.324146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] command line args: [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 08:56:51.324229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.324437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.324515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 08:56:51.324664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ================================================================================ {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 08:56:51.324743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.324900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.324977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] admin_token = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.325238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] config_dir = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.325468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] config_source = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.325697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] debug = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.325988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.326097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.326181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.326326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.326406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.326512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.326590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.326759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.326837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.326994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.327188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_config_append = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.327381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_dir = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.327656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_file = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.327890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.327993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.328135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.328242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.328378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.329002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.329155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.329258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.329345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.329492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.329572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.329731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.329808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.329995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.330089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.330229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.330305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.330460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.330539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.330690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.330782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.330918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.330996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.331162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.331241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.331404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.331494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.331646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.331799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.331879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.332115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.332305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.332457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.332531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] publish_errors = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.332658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.332735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.332859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.332941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.333059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.333134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.333250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.333332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.333461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.333539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.333655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.333736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] standard_threads = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.333934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.334017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.334259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.334571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] use_journal = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.334767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] use_json = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] use_stderr = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.334962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335065 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] use_syslog = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.335161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 08:56:51.335453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.external = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.335971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.336959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.password = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] auth.token = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.337968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.338894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.339896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.340897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.341903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.342925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.343987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.344098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.344827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.344949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.345967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.346898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.347946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.348944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.349990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.350907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.351867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.352963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.353944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.354962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.355947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.356906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.357870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.358960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.359963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.360994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.external = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.361978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.362940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.password = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] auth.token = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.363885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.364846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.365997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.366943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.367980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.368978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.369865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] role.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] role.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.370996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.371854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.372982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.373951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.374979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.375961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.376940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.377932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.378947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.379902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.380993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.381959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.382918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.connection = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.383783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.384998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.385925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.386981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.387997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80375) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:56:51.388069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.388295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.388545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.388791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.389033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.389277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.389522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.389752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.390020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.390259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.390494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.390730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.390976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.391748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.391996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.392235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.392486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.392717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.392960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.393196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.393537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.393776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.394192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.394543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.394779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.395019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.395259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.395600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.395856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.396096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.396410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.396660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.396895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.397198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.397437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.397716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.397933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] role.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.398183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] role.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.398414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.398661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.398908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.399160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.399401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.399645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.399894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.400134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.400380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.400610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.400913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.401150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.401345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.401588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.402045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.402284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.402533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.402811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.403174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.403425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.403662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.403917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.404169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.404407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.404638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.404866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.405096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.405346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.405600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.405875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.406105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.406339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.406568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.407367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.407603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.408122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.408384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.408626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.408955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.409372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.409804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.410255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.410659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.410909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.411224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.411401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.411652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.411886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.412135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.412497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.412765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.connection = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.413005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.413322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.413493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.413738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.414051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.414337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.414573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.414811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.415056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.415293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.415540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.415774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.416021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.416256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.416500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.416734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.416969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.417201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 08:56:51.417370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80374) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 08:56:51.418410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.418644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.418872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.427408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.444012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.444343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.444570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.454095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.458370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.458639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.475929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.476154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.512423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.515876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.565218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.565458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.568549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.568786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.633117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.633343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.633573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.633794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.634989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.635204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.658732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.658915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.659871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.660003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.660085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.660215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.671662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.672372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.684935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.685860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80375) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.686957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80374) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 08:56:51.705004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.705184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:56:51.705531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.705619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.705722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:56:51.705802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:56:51.706005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 08:56:51.706518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.706854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.707027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.707276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.716549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.716910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.717183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.717450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.717566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.717921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.718337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.718807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.724527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.725993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.729379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.729656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.729960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.730241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.730521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.730777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.731311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.731928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.732114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.732192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.732371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.732630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.732894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.733157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.733425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.733686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.755913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.757419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.757700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.758130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.759630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.759908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.762342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.762670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.762902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.764626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.764901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.765173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.770287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.770571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.772845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.773128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.774717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.775000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.775290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.775565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.775836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.776115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.776381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.776654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.777576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.777950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.778239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.778514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.778781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.779055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.779327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.779605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.793303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.793580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.795919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.796202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.797587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.800353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.800757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.801053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.803496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.803796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.805713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.808570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.808905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.811734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.812267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.812564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.812849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.813642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.813959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.814251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.815853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.816148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.816426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.816706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.816978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.817260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.830944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.831237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.831514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.831774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.834540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.834839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.835142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.835407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.838942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.840507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.840776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.843201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.843630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.843906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.844244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.844527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.844800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.845264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.845665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.850231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.850661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.851099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.851534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.856439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.856718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.856994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.857270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.857549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.857832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.858135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.873079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.873566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.874042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.874136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.874498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.875046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.876045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.876511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.877646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.880627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.883295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.884930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.885214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.885497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.892816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.894479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.896325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.897439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.897727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.898038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.898314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.899528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.902656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.905263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.906404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.906724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.906812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.907005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.907090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.907320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.914093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80375) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.914370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.914576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.914678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.914906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.915369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.915490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.915720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.916001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.916174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.916537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.916634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.916913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.917422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.917422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.917694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80375) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.919292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.919431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.919673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.920192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.928258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.928553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.928832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.937146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80374) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 08:56:51.937422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.937698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.938001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.938282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.938549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.938829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.939097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.939468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.939743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.940556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.940830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.941101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.941359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80374) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 08:56:51.952833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564aadc3b9c0 pid: 80375 (default app) Jul 28 08:56:51.956342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbc4a-54f8-47b2-83a8-e36273142b56 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:51.956632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbc4a-54f8-47b2-83a8-e36273142b56 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:51.956908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbc4a-54f8-47b2-83a8-e36273142b56 None None] PATH_INFO: `/v3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:51.961223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Thu Jul 28 08:56:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:56:51.975725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564aadc3b9c0 pid: 80374 (default app) Jul 28 08:56:59.653178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.653764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.653764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.655104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 2/2] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:56:59.663267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:56:59.663775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:56:59.663869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:56:59.735600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: INFO dbcounter [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] Registered counter for database keystone Jul 28 08:56:59.756968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG oslo_db.sqlalchemy.engines [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80374) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:56:59.763476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writer thread running {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:56:59.947703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG passlib.handlers.bcrypt [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80374) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:56:59.948159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG passlib.handlers.bcrypt [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80374) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:57:00.003953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:00.005590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:00.135520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1/3] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:56:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:57:00.254678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:00.332905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: INFO dbcounter [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] Registered counter for database keystone Jul 28 08:57:00.352572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG oslo_db.sqlalchemy.engines [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80375) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 08:57:00.359055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writer thread running {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 08:57:00.479599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9743244-6f5d-4fcd-b807-cc0078d731b5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:00.483960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:00.484594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.484897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.485168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.493387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:00.545699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:57:00.546073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54139-afd3-4f4d-be3e-8f097a6f0609 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:00.556885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 3/4] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Jul 28 08:57:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 08:57:00.563491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:00.583721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68416fa8-7f80-45f1-a0ad-976554ac4b8c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:00.588273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:00.589400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:00.589682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:00.590016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] PATH_INFO: `/v3/projects/66b6795243644c5d8ae6b99e815d8914` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:00.591514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] RBAC: Authorizing `identity:get_project(project_id=66b6795243644c5d8ae6b99e815d8914)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:00.642914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 08:57:00.643255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114f1f0-66c3-4645-92b9-5c36a8811b01 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:00.648049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 2/5] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Jul 28 08:57:00 2022] GET /identity/v3/projects/66b6795243644c5d8ae6b99e815d8914 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:02.663863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f884-55c1-4d95-9af5-d7ae634d9622 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.664396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f884-55c1-4d95-9af5-d7ae634d9622 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.664396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f884-55c1-4d95-9af5-d7ae634d9622 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.668742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 4/6] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:02.674406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.674818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.674961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.695745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:02.697264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:02.706374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 3/7] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 08:57:02.728206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2f884-55c1-4d95-9af5-d7ae634d9622 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.740411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2f884-55c1-4d95-9af5-d7ae634d9622 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.744648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.745272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.745553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.745828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.754450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.756227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087155f-e7b9-4bb7-b296-f6c6c4d21df0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.765078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 5/8] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 08:57:02.771600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:02.781756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740c38e-82e3-426d-b6f1-61e8b69c9777 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:02.786320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:02.787152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:02.787527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:02.787810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:02.789190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:02.791066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc85636-1eef-4bfe-8ef8-018ce91fec68 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:02.792643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 4/9] 10.222.0.48 () {56 vars in 1198 bytes} [Thu Jul 28 08:57:02 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:04.888747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dfd06-3c43-48bc-8664-ec2b3d06ce1f None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.889285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dfd06-3c43-48bc-8664-ec2b3d06ce1f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.889285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dfd06-3c43-48bc-8664-ec2b3d06ce1f None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.890760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 6/10] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:04.896578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.896991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.897142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.902105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194009fe-b3aa-480b-9d6a-7ce8440f0aaf None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.902394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194009fe-b3aa-480b-9d6a-7ce8440f0aaf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.902676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194009fe-b3aa-480b-9d6a-7ce8440f0aaf None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.904233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 7/11] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:04.909348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:04.909632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:04.909941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:04.916488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:04.917669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:04.952464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG passlib.handlers.bcrypt [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80375) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 08:57:04.952886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG passlib.handlers.bcrypt [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80375) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 08:57:04.998896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 5/12] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:04.999099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:05.001356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:05.022924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.046896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.105269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 8/13] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:05.127378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.148235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b777d2-c960-4837-bf9a-51586065ad35 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.184070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.184726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.185006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.185491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.197726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:05.207114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:05.208006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:05.208791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 6/14] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:05.214248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.236350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad880-65ae-43d3-b903-779bf77d9dd6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.239144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.276368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.277255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.277547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.277827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.299908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.319319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac124ab-0508-47af-a37f-c226f7498fc5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.345252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.346856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.351932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.352558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.352841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.353117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.353859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:05.354180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.355572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.369144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 7/15] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:05.374648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.455323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 9/16] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:57:05.461177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.464402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.565872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c72d0-636b-426f-b599-c84e46df90e6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.590751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19232758-09eb-4301-a35a-562d85515b17 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.601572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.602465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.602747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.603020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.622143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.623626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.625742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.626430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.626713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.626990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.628177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.629629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.631264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 8/17] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:05.636816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:05.641619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:05.642477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:05.643253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:05.643253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:05.643253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:05.643253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:05.643253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:05.648668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.652538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.707245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.741598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b7955-0715-4dc8-8425-d0b147c7a870 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.752222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545980d-02c6-40cc-ae8a-d4f1f5e772e9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.775178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf268123-156a-4739-b91c-d75137575712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.775838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268123-156a-4739-b91c-d75137575712 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.776125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268123-156a-4739-b91c-d75137575712 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.776411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268123-156a-4739-b91c-d75137575712 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.777592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf268123-156a-4739-b91c-d75137575712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.779111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf268123-156a-4739-b91c-d75137575712 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.780717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:05.786240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.786877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.787101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf268123-156a-4739-b91c-d75137575712 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:05.787211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.787477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.788123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:05.788424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.789962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e411d-2b8d-4301-a288-c6f1d2afcaec None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.808015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 11/20] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:05.878225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:05.900160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf268123-156a-4739-b91c-d75137575712 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:05.937826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:05.938519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:05.938806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:05.939092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:05.939906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.domain.id=default, user.id=7b84f6da05694fb8ade9ceaae63ccb08) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:05.940215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:05.941719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d03431-1c98-4792-90f5-05df9fe58b39 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:05.949283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 10/21] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Jul 28 08:57:05 2022] GET /identity/v3/role_assignments?user.id=7b84f6da05694fb8ade9ceaae63ccb08&scope.domain.id=default&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:08.251865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67119f1-783a-4615-99ed-541b8b415e8e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.252862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67119f1-783a-4615-99ed-541b8b415e8e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.253442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67119f1-783a-4615-99ed-541b8b415e8e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.255286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 12/22] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:08.262359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3067c7-a8cf-4f9a-8466-302e57f48dae None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.263211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3067c7-a8cf-4f9a-8466-302e57f48dae None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.263678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3067c7-a8cf-4f9a-8466-302e57f48dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.290867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3d3067c7-a8cf-4f9a-8466-302e57f48dae None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:08.292413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3d3067c7-a8cf-4f9a-8466-302e57f48dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:08.378169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 11/23] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:08.414465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67119f1-783a-4615-99ed-541b8b415e8e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.463410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7258e8-c6be-4a9e-8c80-e4155f39aed2 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.464081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7258e8-c6be-4a9e-8c80-e4155f39aed2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.464593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7258e8-c6be-4a9e-8c80-e4155f39aed2 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.466267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 12/24] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:08.481822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.482527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.483015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.523231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:08.524808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:08.591183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67119f1-783a-4615-99ed-541b8b415e8e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.619744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.663697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.664378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.664892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.665332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.688684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.690636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.712386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 13/25] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:57:08.717492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:08.718406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:08.719910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:57:08.719910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:57:08.719910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:57:08.719910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:08.719910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 13/26] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:08.725153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.742240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:08.745755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:08.884341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68d74e-0634-4da7-a2ea-6c60c59ddb61 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:08.941026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:08.941666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.941970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.944891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.945614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:08.945948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:08.947359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca843c-eae6-4ac8-882b-7b784dfed457 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:08.983957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 14/27] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:08.985625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93683b-0364-48b4-b80f-b8a1ae82ad59 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:08.986878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93683b-0364-48b4-b80f-b8a1ae82ad59 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:08.990248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93683b-0364-48b4-b80f-b8a1ae82ad59 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:08.992895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 15/28] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:08.993973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93683b-0364-48b4-b80f-b8a1ae82ad59 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.014306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b819-6d71-4e79-b32b-089b9680ec4f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.088837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.090010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.090494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.090966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.104014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:09.106997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.154240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93683b-0364-48b4-b80f-b8a1ae82ad59 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.197296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.198232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.198654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.199089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.199962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.201701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598355cb-bbb2-4603-ada8-7aeb420663c3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-791fbe0b-9969-4ace-ab4e-9e3f0973a5d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:09.223168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:09.225137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:09.226918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 14/29] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:09.229372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e77a-46f3-44a2-a0ea-49951536d45a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.229766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e77a-46f3-44a2-a0ea-49951536d45a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.230167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e77a-46f3-44a2-a0ea-49951536d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.256167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0b60e77a-46f3-44a2-a0ea-49951536d45a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:09.258334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0b60e77a-46f3-44a2-a0ea-49951536d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:09.284909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.340817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 15/30] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:09.342545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa03eb1-3cc8-4b2b-885a-6a2bac2e3248 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.343021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa03eb1-3cc8-4b2b-885a-6a2bac2e3248 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.343630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa03eb1-3cc8-4b2b-885a-6a2bac2e3248 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.347064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 16/31] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:09.348693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa03eb1-3cc8-4b2b-885a-6a2bac2e3248 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.381914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 16/32] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:08 2022] POST /identity/v3/projects => generated 301 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:09.383688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.383989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.384274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.412471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:09.414544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:09.504908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 17/33] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:09.506173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.527075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa03eb1-3cc8-4b2b-885a-6a2bac2e3248 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.584557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.585661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.585661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.585950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.586696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:09.587008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.588600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.596878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.610747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 17/34] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:09.612015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.645888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.688585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563ecb-33bd-48aa-ab8c-41ef8cd76bc3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.728643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.729298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.729588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.730340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.730632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.732320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.751647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 18/35] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/roles => generated 225 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:09.752010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2edb-8c62-460c-81ed-c783c68b9226 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.752200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.814551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.815237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.815523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.815803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.816551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.818018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69875906-eef6-4616-bd8d-1f98f1f7e3c3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.921795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4d88-5990-4e46-a219-37079771249d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:09.934310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:09.976606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:09.977518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.977621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.977904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.979148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:09.981012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca49e0-2072-49e0-8df1-643737fcd302 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:09.982613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 19/36] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:09 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff21c4-d246-4e27-ac12-87b08d98eb12 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff21c4-d246-4e27-ac12-87b08d98eb12 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff21c4-d246-4e27-ac12-87b08d98eb12 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 20/37] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb76-bf09-4f76-8ffa-20bab4858d64 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.987837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb76-bf09-4f76-8ffa-20bab4858d64 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.988319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb76-bf09-4f76-8ffa-20bab4858d64 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:09.988319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 21/38] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:09.989464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 18/39] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/roles => generated 219 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:09.990638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574eb76-bf09-4f76-8ffa-20bab4858d64 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:09.997615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de9a5-c132-4684-87b6-0e6b9997399b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:09.997995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de9a5-c132-4684-87b6-0e6b9997399b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:09.998245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de9a5-c132-4684-87b6-0e6b9997399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.021373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9c0de9a5-c132-4684-87b6-0e6b9997399b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:10.024218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9c0de9a5-c132-4684-87b6-0e6b9997399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:10.127362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 19/40] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:10.129109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.129440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.129702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.160009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:10.161289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:10.164593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:10.181775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574eb76-bf09-4f76-8ffa-20bab4858d64 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:10.242323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:10.242994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.243383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.243772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.245208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:10.246817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:10.248456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 22/41] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:10.249253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:10.273151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 20/42] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:10.274076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:10.323237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:10.396285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73677e29-a624-466f-bb9f-27f19a2116ea None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:10.403180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2377-b10d-4da6-9551-5caf853afc3c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:10.434124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:10.434789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.435074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.435351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.436567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:10.438674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:10.440330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 23/43] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:10.441098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:10.451405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:10.452139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.452366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.452644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] PATH_INFO: `/v3/domains/default/users/7b84f6da05694fb8ade9ceaae63ccb08/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.459357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7b84f6da05694fb8ade9ceaae63ccb08, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:10.462142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:10.478303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:10.482951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 21/44] 10.222.0.48 () {58 vars in 1361 bytes} [Thu Jul 28 08:57:10 2022] PUT /identity/v3/domains/default/users/7b84f6da05694fb8ade9ceaae63ccb08/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:10.484227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:10.575316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44736b7-8aa3-4bd7-b49f-6bf7b9e0dc0d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:10.593951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:10.631615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff66b75-5fe1-40cd-a9ea-62fbc23e9597 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:10.634659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:10.635307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.635595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.635877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.636626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:10.638101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee677-1dc0-4052-92b2-651f6828f701 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:10.657863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 24/45] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 28 08:57:10 2022] POST /identity/v3/roles => generated 223 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:10.683673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:10.683673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:10.683673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:10.684053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:10.684641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:10.686244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f1a8f-3ea8-4a62-b37e-facead580780 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:10.719725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 22/46] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:10 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:12.560567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e95d-0da8-45d0-894f-4b37b8393a8b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.561122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e95d-0da8-45d0-894f-4b37b8393a8b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.561122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e95d-0da8-45d0-894f-4b37b8393a8b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.562524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 25/47] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:12.567898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1b8c-0030-437d-a1e2-4ce6bbad1746 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.568343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1b8c-0030-437d-a1e2-4ce6bbad1746 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.568447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1b8c-0030-437d-a1e2-4ce6bbad1746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.588551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a18b1b8c-0030-437d-a1e2-4ce6bbad1746 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:12.589767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a18b1b8c-0030-437d-a1e2-4ce6bbad1746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:12.653619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.686653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 23/48] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:12.710160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76e95d-0da8-45d0-894f-4b37b8393a8b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.815384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76e95d-0da8-45d0-894f-4b37b8393a8b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:12.835612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401b25e-7e20-427c-be54-d5ddd91d4e6e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.835892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401b25e-7e20-427c-be54-d5ddd91d4e6e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.836182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401b25e-7e20-427c-be54-d5ddd91d4e6e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.837578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 24/49] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:12.843798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.844920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.845315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.852779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:12.853437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:12.854858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:12.854858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:12.866231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:12.866411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:12.867467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:12.871422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:12.872365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:12.873084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 26/50] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:12.882113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.913082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:12.949423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 25/51] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:12.972549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:12.994339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb736c4-4856-4741-97f0-cfe1a9f9a1e3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.035076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.035733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.036019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.036297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.037020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:13.037330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.038852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.047717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 27/52] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:13.053254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.080844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec209fbb-f48b-450e-86f8-699612643f83 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.095375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.119873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.120530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.120810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.121086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.122332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.123781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.125428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 26/53] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:13.131237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.152500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.163587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d2664-c22b-4753-b91c-95546a370d54 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.199731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.200380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.200669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.200944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.202165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.203683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.205790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 28/54] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:13.211113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.231690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2f4b-f96a-43d2-8d7a-81eb8a4c001e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.275365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.276007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.276295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.276575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.277274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.278799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.317855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1c20-7c35-4c5b-a85d-8e1c4e0afbfe None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.319933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 27/55] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Jul 28 08:57:13 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:13.366411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.366973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.367266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.367546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.368884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.370486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eda7f-8aeb-467a-afec-ed8c40125051 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.372493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 29/56] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:13.378604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:13.423980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:13.500315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25efe-fa50-40b9-96ef-235c752e82c1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:13.537361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:13.538037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:13.538329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:13.538603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:13.539406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.domain.id=default, user.id=7b84f6da05694fb8ade9ceaae63ccb08) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:13.539723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:13.541189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb9ed2-56f7-4be7-a9a2-b597d6abcad4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:13.549707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 28/57] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Jul 28 08:57:13 2022] GET /identity/v3/role_assignments?user.id=7b84f6da05694fb8ade9ceaae63ccb08&scope.domain.id=default&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:15.405311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345afb-56b6-4e71-91c2-d53b142a67fc None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.405866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345afb-56b6-4e71-91c2-d53b142a67fc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.405866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345afb-56b6-4e71-91c2-d53b142a67fc None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.407248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 30/58] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:15.412473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.412875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.413018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.431481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:15.432695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:15.499309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 29/59] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:15.523686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81345afb-56b6-4e71-91c2-d53b142a67fc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.552012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.637505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81345afb-56b6-4e71-91c2-d53b142a67fc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.672712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.673275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.673563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.673874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.675036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.676486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eadf6-d7e4-4fa4-9692-3cba376e8ee1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.678117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 31/60] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:15.684021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:15.711051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:15.796674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed3725-82be-43a8-8d7c-494c8b46a512 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:15.832414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:15.833050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:15.833333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:15.833614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:15.834351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:15.835829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2354ce3-e99d-47c1-96b4-ba701cc8c8b7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:15.864717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 30/61] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:15 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:18.981308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063205e-b228-44fe-a837-985f110fd2c6 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.982276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063205e-b228-44fe-a837-985f110fd2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.982749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063205e-b228-44fe-a837-985f110fd2c6 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:18.985081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 32/62] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:18.990149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5886f21-457c-4f64-86d8-12f092c12ebc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:18.990581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5886f21-457c-4f64-86d8-12f092c12ebc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:18.990853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5886f21-457c-4f64-86d8-12f092c12ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.018866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d5886f21-457c-4f64-86d8-12f092c12ebc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:19.018866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d5886f21-457c-4f64-86d8-12f092c12ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:19.108254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 31/63] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:19.135591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063205e-b228-44fe-a837-985f110fd2c6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.272282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.279807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063205e-b228-44fe-a837-985f110fd2c6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.316741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a50e93-3f54-4bf9-8975-a47981ea0a72 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.317303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a50e93-3f54-4bf9-8975-a47981ea0a72 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.317754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a50e93-3f54-4bf9-8975-a47981ea0a72 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.319855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 32/64] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:19.328798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284f7b-1112-44fa-b5a8-21b2b272c7ea None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.329738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284f7b-1112-44fa-b5a8-21b2b272c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.330569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284f7b-1112-44fa-b5a8-21b2b272c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.335661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.338888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.338888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.338888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.353870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:19.358044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:19.359025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:19.360429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 33/65] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:19.365632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.370792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-52284f7b-1112-44fa-b5a8-21b2b272c7ea None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:19.372162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-52284f7b-1112-44fa-b5a8-21b2b272c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:19.385923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.464005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 33/66] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:19.465737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c716e-feb0-4071-9d42-cdf9f99bba94 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.466281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c716e-feb0-4071-9d42-cdf9f99bba94 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.466773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c716e-feb0-4071-9d42-cdf9f99bba94 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.468190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 34/67] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:19.470015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46eb296-ce93-45a5-8516-9fb46cead38e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.470432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46eb296-ce93-45a5-8516-9fb46cead38e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.470852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46eb296-ce93-45a5-8516-9fb46cead38e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.472202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 35/68] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:19.480224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4d47-37cf-4525-9258-662601045be2 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.480776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4d47-37cf-4525-9258-662601045be2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.481068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4d47-37cf-4525-9258-662601045be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.503304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-588d4d47-37cf-4525-9258-662601045be2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:19.504520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-588d4d47-37cf-4525-9258-662601045be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:19.525060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dbdc1-01c2-4008-aa3d-d241626906a7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.562646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.563342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.563569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.563851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.564760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:19.564926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.566342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf30d9c-0e97-4bf3-a572-b1b9e7118a19 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.589672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 34/69] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:19.589672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.589672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.589672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.596555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 36/70] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:19.601341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4d47-37cf-4525-9258-662601045be2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.607672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:19.608953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:19.624208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.684846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.706596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 35/71] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:19.712482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.731382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4d47-37cf-4525-9258-662601045be2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.779186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.780755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.781552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.782186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.799670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:19.809174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:19.810077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:19.810918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:19.816863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 37/72] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:19.818047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.829541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9277096-f1e6-4d00-8e62-d45514453a5c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.864885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.865657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.866082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.866458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:19.883534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:19.885412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:19.893372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 36/73] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:19.893372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:19.898715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.950337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f85274-4fdf-40cd-b510-a06f40d92066 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:19.987071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:19.993130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:19.993775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:19.994083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:19.994358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.000835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92a3c-c6eb-44fb-9de7-f8ca725c048e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.005794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:20.010834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:20.011705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:20.012604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 38/74] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:20.013984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.040184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.040761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.041046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.041326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.052652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:20.058580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:20.059466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:20.060217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:20.061054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 37/75] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:20.062021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.121371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c17972-4dbe-4b48-8df9-b9beadccb093 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.132206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.165408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.166075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.166361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.166636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.167373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.167684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.169438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89081eef-d898-4170-9c4c-16e6af4ded0e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.180083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 39/76] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.181061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15f9f-29a1-4729-ae90-a06ccab43802 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.182635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4533d-4b84-4fc8-a42d-e931ba018a45 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.182635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4533d-4b84-4fc8-a42d-e931ba018a45 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.182635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4533d-4b84-4fc8-a42d-e931ba018a45 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.183413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 40/77] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:20.183866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4533d-4b84-4fc8-a42d-e931ba018a45 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.220016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.220661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.220947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.221229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.241728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.288908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4533d-4b84-4fc8-a42d-e931ba018a45 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.291938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.293594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.326218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.326970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.327138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.327415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.328156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.328468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.329896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.338784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 41/78] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.339625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.371564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.412813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 38/79] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:20.413969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.443568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a710-f537-4cac-a90d-5bab7349f6cb None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.459213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.476772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.477409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.477692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.478000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.478720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:20.479029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.480417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc92e2f-6a2f-4e1f-9b15-facf5da33867 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.489074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 42/80] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.490391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.490684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.490962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.508923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:20.510156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:20.515274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f5434-ff12-4b24-9239-f8491b698f17 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.549281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.549881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.550168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.550444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.551602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.553043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.554908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 39/81] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.555660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.579843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 43/82] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:20.580763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.598504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.663804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bad65-b969-4d6b-b1c0-76288e001a30 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.683887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325b47-5455-428a-ad89-14f20a785e65 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.692785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.697779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.698478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.698763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.699043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.700226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.701677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.703351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 40/83] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.704114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.723514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.724162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.724444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.724718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.728317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.728466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.730180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 44/84] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.730956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.811516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9233c2-7d78-470e-9a53-a55b87e59770 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.813006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.844657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d58749-3474-47b4-9b37-4955c5e81836 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:20.854840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.855506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.855799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.856082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.857303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.858824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.861067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 41/85] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:20.861890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:20.887844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:20.888504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:20.888785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:20.889063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:20.946330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:20.970946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:20.972445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:20.977038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92bb5-4a2c-4de7-9765-2ff16d6e0508 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.011181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.011858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.012105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.012385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.013566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.015078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.017251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 42/86] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:21.018168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.072979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.092702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 45/87] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:21.093601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.138465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3ecf-538b-449e-86a8-4b9f5fe22622 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.171201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.171866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.172152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.172431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.191774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.214804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35c11f-7dc7-414e-90eb-6e0b79368670 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.249893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.251023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.251315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.251597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.252400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:21.252718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.253408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.254201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.254920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.259922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 46/88] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6273caafd8e64714b134898490672923 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:21.260750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.300377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.400520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 43/89] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:21.402067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.415491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b1d2-48fa-482b-a926-68d248da1e99 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.455009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.455644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.455927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.456200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.467534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.523009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9815f-3040-46ad-bd35-b1f856ada97d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.531744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.533341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.561135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.561765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.562076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.562355 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.563578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.565044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.580955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.604988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 44/90] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:21 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:21.605830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.674076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 47/91] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:21.675034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.735600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:21.765809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b480e80-a268-4599-be50-10c95dd7e6de None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.814400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a5b0-7bc9-494c-8097-174e6a233915 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:21.817979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.818779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.819140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.819578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.821461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.824213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.829356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 45/92] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:21.829578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.866146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:21.866839 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:21.867121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:21.867399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:21.868587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:21.870088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:21.872067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 48/93] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:21.872852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:21.960491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.018926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf020c-740b-48f4-9276-de76e038aaa4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.031414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8539-4a61-46d5-81eb-9e99f3ba6944 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.035153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.076729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.078895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.079202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.079488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] PATH_INFO: `/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.080873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] RBAC: Authorizing `identity:get_project(project_id=c1a26c6bb06b4b45a76c7b5441e85228)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.082559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.084391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 46/94] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.085292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.094588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.095946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.096453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.096929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.098063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=7b84f6da05694fb8ade9ceaae63ccb08) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.098551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.100373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.114453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 49/95] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:21 2022] GET /identity/v3/role_assignments?user.id=7b84f6da05694fb8ade9ceaae63ccb08&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.115703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:22.238408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dd8dc-c91a-4d7d-8694-dac2b17cee3b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.269531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6fa24-bf6f-4d6f-871a-bea5652743df None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:22.324952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.325604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.325913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.326193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.327007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=c1a26c6bb06b4b45a76c7b5441e85228, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.327317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.329379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9323e83-6c6b-4ce7-9a97-e0b6175d28f4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.344548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 50/96] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=c1a26c6bb06b4b45a76c7b5441e85228&role.id=6273caafd8e64714b134898490672923 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:22.350280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:22.358664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:22.359384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:22.359665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:22.359940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:22.360764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:22.361199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:22.362628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77187f4-248f-46c7-8d80-2a0c82420012 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:22.375729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 47/97] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:22 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:23.980363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ea64-5ca3-4035-987b-6040daa65105 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.980864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ea64-5ca3-4035-987b-6040daa65105 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.980934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ea64-5ca3-4035-987b-6040daa65105 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:23.982462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 51/98] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:23.998619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:23.999130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:23.999202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.026346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:24.027546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:24.127179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 48/99] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:24.161397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ea64-5ca3-4035-987b-6040daa65105 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.209916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.299605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ea64-5ca3-4035-987b-6040daa65105 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.345334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.346011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.346294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.346573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.358765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:24.364206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:24.365012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:24.365746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 52/100] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:24.371479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.507324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492c7f9-5722-46cb-b427-359f0749ecb3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.549541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.561143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.561774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.562088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.562366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.563640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:24.566295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.567693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.576770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 49/101] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:24.582914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.713264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b46a4-9092-4222-8cc5-dbd33a2520d8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:24.720766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:24.762762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:24.763342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.763623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.763901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.765135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:24.766681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d640aa-80c2-480e-8cf4-331d0cd9ac82 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:24.768494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 53/102] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:24.773761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:24.847409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a59d-77b1-434a-87d1-52f8eaa2c43d None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.847747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a59d-77b1-434a-87d1-52f8eaa2c43d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.847964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a59d-77b1-434a-87d1-52f8eaa2c43d None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.849613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 54/103] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:24.855184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:24.855474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:24.855756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:24.891018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:24.892270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:24.892373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b26b-4181-494a-a736-489be78d8206 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.007653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.008573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.008701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.009046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.010369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.011859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.013677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 50/104] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:24 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:25.019660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.037883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 55/105] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:25.070245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.147969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.220145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6702c-b847-4ba2-9ba8-d3bd5b230a42 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.248967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727ec98-1612-45e9-834a-f69c9143d57f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.270612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.291297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.292003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.292296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.292577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/e9282f610393452ab15e259a75209eb8/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.299366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, user_id=e9282f610393452ab15e259a75209eb8, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.300854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0af992-8746-418c-85a3-ba8d487c63ac None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.328296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 51/106] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:25 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/e9282f610393452ab15e259a75209eb8/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:25.338851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.339561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.339856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.340140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.360667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:25.374044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:25.376888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:25.378859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 56/107] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:25.382146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2379-81ec-4bd4-bfb2-ddd3a8860d9c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.382146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2379-81ec-4bd4-bfb2-ddd3a8860d9c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.382146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2379-81ec-4bd4-bfb2-ddd3a8860d9c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.382146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 52/108] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:25.384815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.386478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.386765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.387044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.427325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:25.428985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:25.508694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6801cc6-7794-43fb-a82e-3cf5f7f2b146 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.549882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.551059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.552243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.552545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.553728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:25.553728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.555528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dcbb9-5f7f-45c9-88aa-f1afb2fa85f5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.567355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 57/109] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:25.568652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdea1-1ac0-4dc3-904d-b4489b7b5300 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.568945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdea1-1ac0-4dc3-904d-b4489b7b5300 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.569364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdea1-1ac0-4dc3-904d-b4489b7b5300 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.570547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 58/110] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:25.571608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2470-3101-43d9-bb47-b42eb8ca9e72 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.572476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2470-3101-43d9-bb47-b42eb8ca9e72 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.572476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2470-3101-43d9-bb47-b42eb8ca9e72 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.573750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 59/111] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:25.575356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 53/112] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:25.578031 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f9f0-bd9c-484f-bf69-41e06775956f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.578383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f9f0-bd9c-484f-bf69-41e06775956f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.578606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f9f0-bd9c-484f-bf69-41e06775956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.579557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.606565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a137f9f0-bd9c-484f-bf69-41e06775956f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:25.608046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a137f9f0-bd9c-484f-bf69-41e06775956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:25.623957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.654813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.692100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 60/113] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:25.697997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8d51-7f49-4a51-ab24-ab4143693d7e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.697997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8d51-7f49-4a51-ab24-ab4143693d7e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.697997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8d51-7f49-4a51-ab24-ab4143693d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.720205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b98f8d51-7f49-4a51-ab24-ab4143693d7e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:25.721400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b98f8d51-7f49-4a51-ab24-ab4143693d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:25.735200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac9509-8a79-49fc-8b37-d9055b690225 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.772414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.773064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.773354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.773626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] PATH_INFO: `/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.788493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] RBAC: Authorizing `identity:get_user(user_id=59c8e6137c7c4ed8b8954bda19c3dfb9)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:25.790310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:25.792161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 54/114] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9 => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:25.793144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.803660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 61/115] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:25.806229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3442-10bb-4387-8860-53f36eeb53b7 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.806964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3442-10bb-4387-8860-53f36eeb53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.807426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3442-10bb-4387-8860-53f36eeb53b7 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.809385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 62/116] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:25.810560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c3442-10bb-4387-8860-53f36eeb53b7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.894895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.905429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59a63-55fc-48d0-8014-0eefb1118d1f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.907842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:25.932443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c3442-10bb-4387-8860-53f36eeb53b7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:25.945590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.946276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.946563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.946846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.957435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:25.962464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:25.963290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:25.964028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 55/117] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:25.967013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:25.970451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:25.971302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.971746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.972188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:25.982829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-05019a1a-f52a-43bc-9e1c-60a9a1bd4cdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:25.988743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:25.989535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:25.990313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:25.992157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 63/118] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:25.993434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:25.993891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:25.994306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.011957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:26.013145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:26.080024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71109164-74a4-483f-ae52-159bfd11638d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.082814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 64/119] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:26.089921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.123472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.124094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.124375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.124650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.159146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.161913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.200009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b75d27-005c-42b4-95b7-72abd8837571 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.202894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.204931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.234166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.234853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.235136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.235409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.236543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.238012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.244172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 65/120] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.244172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.320565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 56/121] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:26.321639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.358363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabc888-2ea0-4b5f-b151-98ca448b2003 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.400056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.400908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.401696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.402209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.402723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.403636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:26.403863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.405259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.407532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.423653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 66/122] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.423653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.464085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf59f0-a0f3-4acb-8c94-5e435426482c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.509971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.511289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.512107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.512770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.514190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:26.514814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.518224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.535008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 57/123] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.536446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.560174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4ec48-ac06-427e-84dd-5c80cc385765 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.605351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.606316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.606768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.607051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.619240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:26.625558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:26.626667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:26.627511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 67/124] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:26.628264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.662095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67addab-d2e7-4090-9e8d-ca720e4e8498 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.685583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.696369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.697581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.698360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.698838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.699187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.699961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.701442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.732718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 58/125] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:57:26 2022] POST /identity/v3/projects => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:26.733792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.755888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e0c0-3fe8-470e-a600-e7095ab77613 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.797252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.798139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.798510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.798888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.800405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.802098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.804054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 68/126] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.805252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.896693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fb9a-f5fa-4d30-b2e4-1e61e341c3e2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.927931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6bae-f3af-4543-aea4-929a3546c0cd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:26.957646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:26.960306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.961499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.962493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.962493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.966406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.968899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 59/127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.969815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:26.971301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:26.971963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:26.972242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:26.972731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:26.974731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:26.976320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:26.978805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 69/128] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:26.978805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.032548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.102461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec04c-7e37-43f8-94e2-6f8f688f9378 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.117648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faa58b-9ea0-4cc3-8a0f-2dcef32a7785 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.155051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.155237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.155613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.155805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.155919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.156161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.156297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.156624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.156891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:27.157197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.157621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=59c8e6137c7c4ed8b8954bda19c3dfb9) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:27.158047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.158608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.159619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.165869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 70/129] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/role_assignments?user.id=59c8e6137c7c4ed8b8954bda19c3dfb9&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.166862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.179419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 60/130] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.180935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.299097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.340677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f6869-8cd4-46de-ae88-fca0c0be3d9f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.351209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.407398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.408715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.409174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.409785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] PATH_INFO: `/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.411844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=c1a26c6bb06b4b45a76c7b5441e85228)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.413820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.415792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 61/131] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.415946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c107a-23f5-4927-bcac-2b10e9bc1119 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.416885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.470673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.471459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.471808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.472153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.473573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.475325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.478554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 71/132] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.479190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.555187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdad35-dad6-40d6-91b3-1295137df7f9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.602896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.603532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68a392-61b3-4fb6-916e-773fee9863a0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.603642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.604013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.604354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.605851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.607538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.611219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 62/133] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.616390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.641169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.660772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.662055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.663487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.663965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] PATH_INFO: `/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228/users/e9282f610393452ab15e259a75209eb8/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.666218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1a26c6bb06b4b45a76c7b5441e85228, user_id=e9282f610393452ab15e259a75209eb8, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.668520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11939671-95e7-4cac-a65d-836f17d41704 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.671319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:27.689504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 72/134] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:27 2022] PUT /identity/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228/users/e9282f610393452ab15e259a75209eb8/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:27.691668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4000cfd-79e8-47f5-934c-ec3d39de9bca None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.692588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4000cfd-79e8-47f5-934c-ec3d39de9bca None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.693374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4000cfd-79e8-47f5-934c-ec3d39de9bca None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.695929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 73/135] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:27.697342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4000cfd-79e8-47f5-934c-ec3d39de9bca None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.743964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece545b-48c5-4245-b1cd-edc744e02ce6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.791531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.792234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.792534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.792850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/e9282f610393452ab15e259a75209eb8/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.795477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, user_id=e9282f610393452ab15e259a75209eb8, role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.797145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e75afa-e9ea-4c12-9a55-ad6d40042b4a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.813294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 63/136] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:27 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/e9282f610393452ab15e259a75209eb8/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:27.814792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.815092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.822072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.853879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:27.853879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:27.862447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4000cfd-79e8-47f5-934c-ec3d39de9bca None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:27.908551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:27.909627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:27.910129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:27.910711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:27.912203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:27.914094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:27.916183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 74/137] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:27 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:27.925762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:27.956752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 64/138] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:28.006097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.017739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.074259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.150766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2e03-621b-4312-91bf-371d1d6411ce None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.214111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.214726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.215009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.215285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/7b84f6da05694fb8ade9ceaae63ccb08/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.218453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, user_id=7b84f6da05694fb8ade9ceaae63ccb08, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.220444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.230812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d03e4-e24b-46e2-8a6d-29a02a63f357 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.247180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 75/139] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:27 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/7b84f6da05694fb8ade9ceaae63ccb08/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:57:28.285411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.286188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.286481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.286764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.305636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:28.310890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:28.311791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:28.312634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 65/140] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:28.319527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.478347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b790c-dcfe-45fe-a6d8-c0e81992cdd5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.496921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.530626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.531861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.532154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.532432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.533182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:28.533487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.534920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.544762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 76/141] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:28.553271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.683410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:28.694534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd14af2-0e0c-4403-893b-28da0c62cee8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.730767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.731343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.731628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.731909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:28.733100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:28.734793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:28.736576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 66/142] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:28.742022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:28.894290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eeb12-f1f7-4357-ad4a-a26e28639eef None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:28.943626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:28.944264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:28.944548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:28.944825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.002648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.028654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.030165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.150369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 77/143] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:29.155985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.292885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb4755-c1ef-4a1d-ad7e-35c1d3c21952 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.338544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.339577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.340075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.340555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.341997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.343717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c46368-6068-4a5c-a57a-f27c6b57c5c5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.345774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 67/144] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:29.351674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.460548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.494846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0bf8d-9fea-4736-9cb6-75b369e1e628 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:29.547788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:29.548339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.548622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.548895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.549705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:29.550040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:29.551517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013d84f-ce2a-4f44-ba5f-cd18ebaa1ff6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:29.558112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 78/145] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6273caafd8e64714b134898490672923 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:29.625978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114561fa-e212-413a-8832-d8324e32522b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.626644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114561fa-e212-413a-8832-d8324e32522b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.627133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114561fa-e212-413a-8832-d8324e32522b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.628759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 68/146] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:29.643703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:29.644016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:29.644268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:29.671620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:29.673430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:29.780959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 79/147] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:29.824770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114561fa-e212-413a-8832-d8324e32522b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:29.876480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:29.993247 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114561fa-e212-413a-8832-d8324e32522b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.034379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.035037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.035318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.035594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.036759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.038267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe60f6-117d-4b78-887c-1ad256fa9d34 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.039963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 69/148] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:30.045505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.153690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.205270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d7764-b628-4dae-a1a6-8412739bb6dc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.244198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.244838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.245116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.245388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.246106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:30.247559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8d3b5-abac-4e22-bca2-15819bddaf03 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:30.275972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 80/149] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:30 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:30.584057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c181-94b7-489b-b54e-08aad9c315a9 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.584580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c181-94b7-489b-b54e-08aad9c315a9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.584580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c181-94b7-489b-b54e-08aad9c315a9 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.586006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 70/150] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:30.591445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.591892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.592106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.613107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:30.614424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:30.719574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 81/151] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:30.741918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.742119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d0df0-c4b4-46be-933d-3f7849b83279 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.742205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d0df0-c4b4-46be-933d-3f7849b83279 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.742467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d0df0-c4b4-46be-933d-3f7849b83279 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.743973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 71/152] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:30.749493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.749774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.750747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.769911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:30.771270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:30.831165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.887164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 72/153] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:30.902966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60006add-9a93-40ea-90a0-7677c0a1e089 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:30.932811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:30.948205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:30.949346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:30.949626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:30.950314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:30.964399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:30.970965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:30.971887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:30.972724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:30.973466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:30.974107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 82/154] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:30.981302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.079325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba576f-a117-4931-86ab-18aaff8fabb3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.119194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.119434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45913f-949e-4af2-8b4a-ae043fffc3b6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.120235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.120662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.121070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.132586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-70b4d5c4-a5c3-46aa-8bb2-7470c073e4fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:31.138357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:31.139378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:31.140369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:31.141076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.141146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 73/155] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:31.141146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510429-929e-4c9f-b019-403743b7f707 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.141326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510429-929e-4c9f-b019-403743b7f707 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.141751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510429-929e-4c9f-b019-403743b7f707 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.143341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 74/156] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:31.149805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd510429-929e-4c9f-b019-403743b7f707 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.170481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.171497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.172545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.173030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.174104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:31.174606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.176512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafdac-9453-4224-9abd-50ac75d95ee6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.187126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 83/157] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:31.188897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.189426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.189936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.212829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:31.214461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:31.270425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.287604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd510429-929e-4c9f-b019-403743b7f707 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.299951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 84/158] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:31.300932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.328854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.329916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.330423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.330894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.331807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:31.332403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.334756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.345063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 75/159] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:31.346268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.442645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55fb1-77f6-43d9-9def-7ba8be1ef069 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.450826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.482292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.483093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.483214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31483ffd-5c9d-4747-8fa1-69936af2b972 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.483458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.483806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] PATH_INFO: `/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.485096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] RBAC: Authorizing `identity:get_user(user_id=59c8e6137c7c4ed8b8954bda19c3dfb9)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.486655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.488674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 85/160] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:31.489452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.538911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.539577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.539859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.540131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.556871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-322135af-7d1a-4f3b-b32b-1f5c68b73abb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:31.564407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:31.565395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:31.566195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:31.567275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 76/161] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:31.569862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210913-0199-46f4-9905-36599d4d6544 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.569862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210913-0199-46f4-9905-36599d4d6544 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.569862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210913-0199-46f4-9905-36599d4d6544 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.574690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 77/162] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:31.575531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d210913-0199-46f4-9905-36599d4d6544 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.592366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.638298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabed000-6c12-41fb-8b0f-348900031727 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.687537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.688054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.688333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.688607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.689980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.691624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.693624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 86/163] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:31.694447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.714320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d210913-0199-46f4-9905-36599d4d6544 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:31.776017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:31.777269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.777750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.778272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.779732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:31.781458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a0f59-d975-4a29-a170-a105434d9708 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:31.784891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 78/164] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:31.786479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:31.786996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:31.787469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:31.799616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.811605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:31.813966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:31.895731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 79/165] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:31.897682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:31.908078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:31.960979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1923d08-2cc8-4e2a-b041-da9f38263516 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.027901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.028546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.028830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.029108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.029870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:32.030378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:32.032085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:32.040047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f39e9-246c-4af2-9852-611fa42d3848 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.041339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 87/166] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:32.042562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.115191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.116217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.116702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.117167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.178289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c21002-3948-4b53-98c0-8635877c401d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.182210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:32.256958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:32.258772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:32.270554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.271477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.271903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.272326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/59c8e6137c7c4ed8b8954bda19c3dfb9/roles/a65333a7deba49b9988b74c07a81dd94` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.279345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, user_id=59c8e6137c7c4ed8b8954bda19c3dfb9, role_id=a65333a7deba49b9988b74c07a81dd94)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:32.281036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:32.290218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:32.311979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 88/167] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:32 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/users/59c8e6137c7c4ed8b8954bda19c3dfb9/roles/a65333a7deba49b9988b74c07a81dd94 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:32.313600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.379518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 80/168] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:32.380622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.491502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970d19-2a50-46d7-9b9e-3d55c03a5549 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.514343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:32.530047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.530875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.531802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.532178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] PATH_INFO: `/v3/users/e9282f610393452ab15e259a75209eb8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.533577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] RBAC: Authorizing `identity:get_user(user_id=e9282f610393452ab15e259a75209eb8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:32.535217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:32.542168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 81/169] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:32 2022] GET /identity/v3/users/e9282f610393452ab15e259a75209eb8 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:32.542168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.588458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813c31-2179-46f7-ac7f-8ead2a5afe78 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.638726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.640361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.641338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.646524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.662294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:32.667106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e946-c291-4e79-8152-b922a30d25c1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:32.672734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:32.673625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:32.674438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:32.678241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 89/170] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:32.679299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.717706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:32.719240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:32.719807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:32.720891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:32.721682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:32.723721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:32.725544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:32.727502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 82/171] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:32 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:32.729770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:32.820104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:32.908731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39ec6-bacb-48c3-bc48-d9bce807fefd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.038058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1aef8-62ed-48a3-b72f-90d20a3f2466 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.046552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.047251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.047530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.047803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.103013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.104459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.105188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.105944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.107514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:33.108261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.111246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.127971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 83/172] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.129932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.138078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.139557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.167088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.288816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 90/173] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:33.289680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.339528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba153b5-f35e-47b2-b228-30b46cc09b13 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.344075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.402199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.403598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.404298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.404967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.406690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:33.407472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.410427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.421971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 84/174] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.423794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.502852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3336-5d28-4169-b5b7-c38d58700e75 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.580025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.615255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e1a2d-47d3-4b02-830f-c424f14ac0d8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.616438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.617682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.618384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.619050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] PATH_INFO: `/v3/users/7b84f6da05694fb8ade9ceaae63ccb08` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.623278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] RBAC: Authorizing `identity:get_user(user_id=7b84f6da05694fb8ade9ceaae63ccb08)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.630810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.634620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 91/175] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/users/7b84f6da05694fb8ade9ceaae63ccb08 => generated 247 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.640294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.679580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.680902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.681438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.682006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] PATH_INFO: `/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.684130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=c1a26c6bb06b4b45a76c7b5441e85228)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.686989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:33.690925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 85/176] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/projects/c1a26c6bb06b4b45a76c7b5441e85228 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:33.698132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:33.772598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:33.892904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671d39-5ea4-4cbe-a2bd-9c61297f32e4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.901433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e957698-2458-469a-913d-2c6979a5b9d4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:33.949406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.950080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.950360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.950632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.986987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:33.987630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:33.987910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:33.988188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:33.988992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=c1a26c6bb06b4b45a76c7b5441e85228, user.id=e9282f610393452ab15e259a75209eb8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:33.989302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:33.990794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accc5c8-dbff-466e-9633-c8c6b335116d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:34.004252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 86/177] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/role_assignments?user.id=e9282f610393452ab15e259a75209eb8&scope.project.id=c1a26c6bb06b4b45a76c7b5441e85228&role.id=6273caafd8e64714b134898490672923 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:34.005524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903b58-5fda-4a51-92c5-fde2f4f34305 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.005813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903b58-5fda-4a51-92c5-fde2f4f34305 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.006112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903b58-5fda-4a51-92c5-fde2f4f34305 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.008449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 87/178] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:34.016471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.016786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.017009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.058279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:34.060858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:34.078279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:34.091878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:34.093398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:34.220061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 88/179] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:34.260046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:34.319744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:34.358628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 92/180] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:57:34.369866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:34.480913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51305b5-0159-4a59-a76d-f295a3197777 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:34.503397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5086e-8691-4fdc-9efa-5fd5ab004428 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:34.505683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:34.546204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:34.547196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:34.547525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.547865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.548144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.548415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.548608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.549257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.550859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:34.553338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22877cbc-b429-450d-9289-9a175dd39625 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:34.555658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 93/181] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:34.557171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbfaa-a335-4820-9e9e-2347c011fbfe None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.557674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbfaa-a335-4820-9e9e-2347c011fbfe None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.558159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bbfaa-a335-4820-9e9e-2347c011fbfe None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.559600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 94/182] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:34.567185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.567857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.568484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.571186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:34.577570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:34.579841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:34.580650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:34.581636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 89/183] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:34.585915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:34.603072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:34.604672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:34.687365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 95/184] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:34.689163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:34.812146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:34.820294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:34.825769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc912b-d721-4bca-ae9d-2657426a7d75 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:34.830596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a50b-237d-431c-8526-14f58491a9ff None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:34.871317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:34.872288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.873486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.873765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.874702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=7b84f6da05694fb8ade9ceaae63ccb08) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:34.875810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:34.877272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:34.883467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 90/185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/role_assignments?user.id=7b84f6da05694fb8ade9ceaae63ccb08&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:34.883601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:34.883990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.884170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:34.884321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.884602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.885319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:34.885624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:34.895036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaff5b-6cdd-4cd1-b41b-6181f3ef2a31 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:34.923982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 96/186] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:34.925508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448cd-e8a6-44d3-bdf3-0b6788746307 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:34.925809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448cd-e8a6-44d3-bdf3-0b6788746307 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:34.926100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448cd-e8a6-44d3-bdf3-0b6788746307 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:34.930248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 97/187] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:34.930248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27448cd-e8a6-44d3-bdf3-0b6788746307 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.092148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a43a-8e35-421b-8c1f-6333dbea382e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.106995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27448cd-e8a6-44d3-bdf3-0b6788746307 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.137274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.137990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.138247 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.138524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.148904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.149567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.149921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.150301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.151039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-947527bd-7cbb-4e6c-87e1-094e3f3eeb0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:35.157729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:35.158663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:35.159452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:35.161436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 91/188] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:35.162903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.163424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.163895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.174719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.176232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.178533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 98/189] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:34 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:35.182694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.183511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:35.184717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:35.194933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.197619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.254912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 92/190] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:35.255766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.322344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9857e1-2810-4409-b48f-7e6c6c08dc77 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.357874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.358442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.358723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.358994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.359917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02ff8a-35e1-4da9-bdeb-de416e4f4e91 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.360798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:35.361105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.362874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.376032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 99/191] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:35.376032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.401391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.402171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.402481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.402754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.418844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.450352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.478872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.480812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.487138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d0c5-0582-4905-9b94-9e2d381773cc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.524224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.524878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.525161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.525438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.527059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.528911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.531570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 100/192] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:35.534720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.592807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 93/193] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:35.594529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.644890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.648241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8d49-79ee-460d-8367-d45d6dc029a8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.685258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.686304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.686964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.687095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.688374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.689866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.691637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 101/194] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:35.692253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.704044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.705060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b835a8-8868-45e1-b68b-fc772db1c092 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.741699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.742356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.742637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.742915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.743597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.745050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e89762-7123-49b5-980c-c15b0d996fee None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.765821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 94/195] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:35 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:35.767101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b690-c8db-4ac1-b08d-55edafede6c0 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.767384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b690-c8db-4ac1-b08d-55edafede6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.767654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b690-c8db-4ac1-b08d-55edafede6c0 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.768916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 95/196] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:35.769649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b690-c8db-4ac1-b08d-55edafede6c0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.812810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74584-b640-4b68-b628-5588b95c3719 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.851650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.852531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.852787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.852997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.854597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:35.856095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff1504-2a55-4966-bb81-1fb603ae66d8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:35.863529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 102/197] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:35.863529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.863529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.863529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.880296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:35.881544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:35.915921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:35.927077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b690-c8db-4ac1-b08d-55edafede6c0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:35.967937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 103/198] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:35.971730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:35.972362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:35.972641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:35.973139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:35.974733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:35.984300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.063546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.065244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.093574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21854-51e5-4466-a73e-8c0ad67cf736 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.139577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.140221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.141036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.141312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.145872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=a213d9870a7e49bbaf00cb4ef34cb273) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:36.145872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.145872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.150173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 104/199] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/role_assignments?user.id=a213d9870a7e49bbaf00cb4ef34cb273&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.151094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.184465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 96/200] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:36.185018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.195201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.242375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.276534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0508984-68a7-43b6-998b-1628a7f12ec0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.322291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.322291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.322291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.322291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.345053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:36.350678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:36.351560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:36.352302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 105/201] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:36.362749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.371158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6e90-7efa-4574-a5d9-5d441f31174b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.409755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.410436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.410717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.410990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.417808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.419376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.421186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 97/202] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.426963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.473297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff69d2-a78c-4fcd-bd0d-2a279253ce8f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.510377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.516754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.530362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.531044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.531479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.531755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.532482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:36.532783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.534199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.550323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 106/203] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.556915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.585208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e71ab-ca47-46d5-aa5b-721702d5d7ee None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.694707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.695347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.695630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.695908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.696711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=a213d9870a7e49bbaf00cb4ef34cb273) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:36.697019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.698567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.706571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102fab-5267-4bec-8cc0-726d6ed3c95f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.711943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 98/204] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/role_assignments?user.id=a213d9870a7e49bbaf00cb4ef34cb273&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:36.752525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.753205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.753697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.754036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] PATH_INFO: `/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:36.755503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] RBAC: Authorizing `identity:get_user(user_id=59c8e6137c7c4ed8b8954bda19c3dfb9)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:36.757569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:36.759707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 107/205] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/users/59c8e6137c7c4ed8b8954bda19c3dfb9 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:36.765194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:36.925498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b6173-f2b6-44cf-8930-dfd78f907de1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:36.965303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:36.974133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:36.974822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:36.975107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:36.975376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:37.047829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:37.049298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:37.150671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 99/206] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:37.156775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:37.283151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:37.343176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1d32-5c0a-4da9-8a7b-aa8fbe1fa9ce None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:37.385211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:37.385896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:37.386181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:37.386457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:37.387641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:37.389113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b0f6f-99da-4ab8-9759-c52ce4c76542 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:37.391375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 108/207] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:37 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:37.397684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:37.439778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:37.550906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b22ec-9027-4c93-af94-d2671fc09e5b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:37.611187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:37.611728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:37.612011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:37.612288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:37.613070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8, user.id=59c8e6137c7c4ed8b8954bda19c3dfb9) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:37.613376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:37.614837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691381e-7fc0-42a3-a40c-c9ef4dc93b3b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:37.626608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 100/208] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:37 2022] GET /identity/v3/role_assignments?user.id=59c8e6137c7c4ed8b8954bda19c3dfb9&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:38.223763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd86ea-1532-4f02-bae1-58975b946331 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.224746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd86ea-1532-4f02-bae1-58975b946331 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.225245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd86ea-1532-4f02-bae1-58975b946331 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.226882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 109/209] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:38.232452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.232951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.233028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.253907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:38.255465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:38.312200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd770f-0262-49bc-9051-5441a1dcfc78 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.313537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd770f-0262-49bc-9051-5441a1dcfc78 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.314046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd770f-0262-49bc-9051-5441a1dcfc78 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.316485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 110/210] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:38.321962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.322513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.322992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.330463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 101/211] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:38.345517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:38.347024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:38.354023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:38.382187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:38.420993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 111/212] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:38.446966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:38.482834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98644ca6-d14c-4613-8776-9e94aa7231c0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:38.509204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:38.545497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:38.546652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.547659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.548125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.558099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2e60-249b-4dff-8588-e1569d9c43b7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:38.567042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:38.577180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:38.578311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:38.579096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:38.579716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 102/213] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:38.588998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:38.599980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:38.600954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.601412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.601891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.603273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:38.604968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:38.606821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 112/214] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:38.613080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:38.720012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6667bc-8fb2-4a85-b819-69be965cc216 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:38.746824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:38.748983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:38.760355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:38.761252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.761909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.762124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.762930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:38.764587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:38.768195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843487f-098d-4d1f-8347-5c8acbec6aa3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:38.790183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 113/215] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:38 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:38.808908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:38.809867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.810316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.810735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.811673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:38.812128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:38.813732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ca16f-6982-4b56-b06b-ee44242e1ad7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:38.824083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 103/216] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:38.830109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:38.893804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f766a-73e1-4221-abf1-802e8c56e02f None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.894118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f766a-73e1-4221-abf1-802e8c56e02f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.894395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f766a-73e1-4221-abf1-802e8c56e02f None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.895784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 104/217] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:38.901797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.902112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.902388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:38.922492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:38.923733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:38.958778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c2c1e-d96d-43b4-bdbb-47ac5d1f847e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:38.997188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:38.997809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:38.998127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:38.998453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] PATH_INFO: `/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.000660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 105/218] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:39.012588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] RBAC: Authorizing `identity:get_user(user_id=5b48f004a9cc4cdda87f3004bed5f7e5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.014228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.016774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 114/219] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:38 2022] GET /identity/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.022475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.023949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.077248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.122773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.146422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428eedc-8e72-4448-952b-27becac45382 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.152053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1621924-cc2e-4998-8d66-f46fffd8ec81 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.193167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.194558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.194688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.196122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.196246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.196321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.196386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.196386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.208074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7f9aeccb-ad09-4029-b7f2-94f6b4f5f600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:39.213253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:39.214398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:39.215379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:39.216605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 115/220] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:39.217687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e7f-6576-4d35-bc51-a3bf8824a262 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.218090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e7f-6576-4d35-bc51-a3bf8824a262 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.218377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e7f-6576-4d35-bc51-a3bf8824a262 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.220609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 116/221] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:39.227231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1e7f-6576-4d35-bc51-a3bf8824a262 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.278627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.281476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f40b4-4065-4885-bd55-c36c962e5b96 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.362984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.365771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.368242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1e7f-6576-4d35-bc51-a3bf8824a262 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.393192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 106/222] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:39.398669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.398972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.399256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.420696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:39.422246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:39.433263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.434682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.435471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.436221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.438414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:39.439258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.442131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.458196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 117/223] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:39.467356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.505920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 107/224] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:39.505920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.622453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327408ee-36f8-4c1f-bae8-5f4f7b0f3f1a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.635330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.656113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8a28-614b-4c37-ae14-c34c748174e1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.669083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.670669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.671124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.672217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.674006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.676087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.678788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 108/225] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.680142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.708633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.710190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.710988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.711602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.714173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.717672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.721491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 118/226] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.727753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.780367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.802559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cf82f-2c70-44a9-a600-d566f834a587 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.838610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.839432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.839830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.840188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.851005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:39.856670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:39.858631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:39.860807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 109/227] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:39.862797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.903430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:39.904556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbfa38-195d-48d8-8f9f-ed2ef470cdde None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:39.944578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:39.945649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:39.946159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:39.946625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:39.948069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:39.949894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:39.951818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 119/228] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:39.952794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:39.985573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b6f6-fa28-4f28-9b8b-42f62689f6f0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.021593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.022264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.022548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.022823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.023623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=5b48f004a9cc4cdda87f3004bed5f7e5) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:40.023924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.025378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.036135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 110/229] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/role_assignments?user.id=5b48f004a9cc4cdda87f3004bed5f7e5&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6273caafd8e64714b134898490672923 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:40.036135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.092474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:40.105165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052fb16-dcb0-4455-b803-7a733022ad3d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.139229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf4b63-284d-49ab-8d60-6b109e5736f6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.141399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:40.163511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.164496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.164967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.165442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.166494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:40.167094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.168750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.180341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 120/230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:40.198073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.199872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.200572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.200856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.201133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/a213d9870a7e49bbaf00cb4ef34cb273/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.203762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, user_id=a213d9870a7e49bbaf00cb4ef34cb273, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.205228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.242162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 111/231] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:40 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/a213d9870a7e49bbaf00cb4ef34cb273/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:40.380415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ef493-8c92-4438-be33-5a8a6310f61f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.427301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.428538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.429041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.429551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.434592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.436377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.438509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 121/232] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:40 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:40.447997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.651506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325163-6d68-4c5a-8f66-fa6e17e5207c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.692879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.693460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.693743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.694055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.695239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.696701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25006071-62f5-4cc4-aa01-2ecfa980ef0c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.699671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 112/233] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:40 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:40.706207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:40.748524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:40.873893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d137f5-6029-4399-976f-75bd2126f5e4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:40.925027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:40.925721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:40.926029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:40.926304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/a213d9870a7e49bbaf00cb4ef34cb273/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:40.930550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, user_id=a213d9870a7e49bbaf00cb4ef34cb273, role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:40.932029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a03f6-8026-4d71-8562-0e589e870933 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:40.959726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 122/234] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:40 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/a213d9870a7e49bbaf00cb4ef34cb273/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:41.324773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403754b-226e-40bf-9364-b0c24721c9df None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.325094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403754b-226e-40bf-9364-b0c24721c9df None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.325327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403754b-226e-40bf-9364-b0c24721c9df None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.326734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 113/235] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:41.332668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.332959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.333233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.351381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57041-9e56-43e5-b3f9-29cb6c99fe4b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.352001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57041-9e56-43e5-b3f9-29cb6c99fe4b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.352482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57041-9e56-43e5-b3f9-29cb6c99fe4b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.354431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 114/236] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:41.356089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:41.357298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:41.360320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.360881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.361354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.380718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:41.382378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:41.430464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:41.493747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 123/237] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:41.506472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 115/238] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:41.525324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:41.543999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:41.559875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:41.707895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52363ba0-97d0-4782-b08a-2c0cb37ef420 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:41.754068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc2544-9cfd-4671-b806-28c69bed5e5c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:41.772033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:41.772690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.772964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.773242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.795304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:41.801310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:41.802191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:41.802976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 124/239] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:41.812264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:41.821880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:41.825993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:41.826273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:41.826591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:41.827728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:41.829203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:41.831128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 116/240] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:41.836342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:41.882195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:41.951737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:41.996217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18224e-de5b-4ed2-b088-9cb2f1f20c60 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.007720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9424964-3dbb-46ea-9a7f-4c68da27ed50 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.060428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.061058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.061334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.061604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.062385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:42.062685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.064090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.070862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.071489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.071763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.072031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.072726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.074190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.085815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 125/241] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:42.091952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.117728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 117/242] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:41 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:42.251088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2b44-7081-4dd9-b4ae-2729e455197d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.304044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.304706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.305246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.305519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] PATH_INFO: `/v3/users/acce5d4ebb8244f98da8b96243481f04` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.322977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:42.326966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] RBAC: Authorizing `identity:get_user(user_id=acce5d4ebb8244f98da8b96243481f04)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.328294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0aa09-7d02-4f8d-a910-589759477c55 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.330499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 126/243] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/users/acce5d4ebb8244f98da8b96243481f04 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:42.336183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.459811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:42.486095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232edc6-c02c-4d70-8169-f8b8debc0f46 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.527424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.528045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.528267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.528529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.608785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.610341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.641722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec0b9-08d4-4afe-9a40-96f4e5a85700 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.642268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec0b9-08d4-4afe-9a40-96f4e5a85700 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.646432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec0b9-08d4-4afe-9a40-96f4e5a85700 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.646432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 127/244] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:42.656281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346ba2c-8cb0-40ac-b58b-89869263b068 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.656281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346ba2c-8cb0-40ac-b58b-89869263b068 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.656281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346ba2c-8cb0-40ac-b58b-89869263b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.675809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4346ba2c-8cb0-40ac-b58b-89869263b068 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:42.677570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4346ba2c-8cb0-40ac-b58b-89869263b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:42.722381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:42.747167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 118/245] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:57:42.754138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.758528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 128/246] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:42.779858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37873a-daf7-403d-8ff6-a0600f7acf6b None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.780468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37873a-daf7-403d-8ff6-a0600f7acf6b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.780989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37873a-daf7-403d-8ff6-a0600f7acf6b None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.782885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 129/247] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:42.788248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.789042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.790553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.814502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:42.815476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:42.871400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad0d5-692c-4c4b-94bf-bebbbe77be15 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:42.904871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 130/248] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:42.905652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:42.922645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:42.923315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:42.923650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:42.923982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:42.925225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:42.927334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:42.932094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 119/249] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:42.936888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.012431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.024012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.089755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7688cd-6b34-47f9-a484-9ac2062ef2da None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.092593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960f1d-594f-43d0-8e9e-4185b8e56f00 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.159311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.159971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.160249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.160522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.162261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.162888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.163163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.163432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.164234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=acce5d4ebb8244f98da8b96243481f04) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:43.164542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.167300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.181830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 120/250] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/role_assignments?user.id=acce5d4ebb8244f98da8b96243481f04&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:43.182716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.184287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:43.192553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:43.193679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:43.194833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:43.198861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 131/251] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:43.207077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.310953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1c7ac-90a1-4779-80c5-56c9abfb882d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.328532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.355391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.356024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.356306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.356584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.369988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:43.375970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:43.376850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:43.377639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 121/252] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:43.383454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.502645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1eecd-94e2-4091-a071-29936d3da4e2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.513741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1158f-e020-4bf5-99b1-8d0c8615430d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.541724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.542424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.542674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.544271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.544271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:43.544271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.545497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.555454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 122/253] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:43.561424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.568419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.602444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.622658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.623222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.623500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.623771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.624500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:43.624844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.626276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2192bd-b21c-4204-91c5-75a6c2da5c8a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.644419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 132/254] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:43.667874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656352-3d89-4e00-989f-6fb291f4bc27 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.668182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656352-3d89-4e00-989f-6fb291f4bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.668418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656352-3d89-4e00-989f-6fb291f4bc27 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.674233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 133/255] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:43.675136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4656352-3d89-4e00-989f-6fb291f4bc27 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.684206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a2b54-d89e-49dc-b4e6-0d3a118be64b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.721085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.721731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.722034 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.722310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.723747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.725706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b8894-d312-43a0-9e4f-3436f1911ebd None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.727416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 123/256] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:43.731046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.732384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.732666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.757097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:43.758380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:43.830470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 124/257] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:43.831392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.840412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4656352-3d89-4e00-989f-6fb291f4bc27 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:43.878617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:43.879190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:43.879472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:43.879746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] PATH_INFO: `/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:43.880930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] RBAC: Authorizing `identity:get_user(user_id=5b48f004a9cc4cdda87f3004bed5f7e5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:43.882422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:43.884061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 134/258] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:43.885644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:43.928326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:43.956034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:44.013967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd495-552c-4d6d-98f1-0f0a5d4336e6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.053488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.054170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.054454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.054730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.067316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:44.074126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:44.075024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 135/259] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:44.076204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.118542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27611e-4a6b-4cef-b619-bf50f1b494be None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.193094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05024a-ae71-4559-bcce-8579f971dce8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.203131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.203712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:44.203818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.204102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.204379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.233223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.235029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.235534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.236015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.237406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.239089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.241751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 136/260] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:44.243069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.392873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8195b-d168-4eb6-9fc6-1ed1fdbca338 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.394060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.395800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.482291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:44.490452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.491185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.491470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.491741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.492474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:44.492778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.494201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.523982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 137/261] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:44.524907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.536101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 125/262] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:44.541439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.648288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424622a-c832-48f2-b8d4-c2283b8a35e4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.684980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.685794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.686200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.686605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] PATH_INFO: `/v3/users/a213d9870a7e49bbaf00cb4ef34cb273` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.688279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] RBAC: Authorizing `identity:get_user(user_id=a213d9870a7e49bbaf00cb4ef34cb273)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.689816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.691717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 126/263] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/users/a213d9870a7e49bbaf00cb4ef34cb273 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:44.692582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.702369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:44.726443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:44.848852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9dd3-2df8-4b2c-af0d-5ea63ecbe77a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.877381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955896a2-23a1-4b70-aedd-e034c48513bf None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:44.897790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.898738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.899080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.899364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/5b48f004a9cc4cdda87f3004bed5f7e5/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.901521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, user_id=5b48f004a9cc4cdda87f3004bed5f7e5, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.903278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.942063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 138/264] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:44 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/5b48f004a9cc4cdda87f3004bed5f7e5/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:44.942953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:44.964792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:44.965520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:44.965801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:44.966140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:44.969365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:44.973794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:44.978903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 127/265] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:44.985238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.141682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8db37-1955-4b02-8d2d-c8c4a87724cb None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:45.142264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.162234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.175501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de768cab-1df1-46c3-a032-d160d4f5ea5c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:45.214586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:45.214798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:45.215458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.215570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.215783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.216011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.216114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.216450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.217567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=a213d9870a7e49bbaf00cb4ef34cb273) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:45.218014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:45.220133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7e3c-0712-4ba6-9f17-9eee2b2cf113 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:45.232104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 128/266] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/role_assignments?user.id=a213d9870a7e49bbaf00cb4ef34cb273&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:45.348510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:45.350141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24539b-41b4-4d36-bbe5-e1048adc8556 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:45.392353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1d04-0370-4486-9e3b-958125cc7044 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.392634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1d04-0370-4486-9e3b-958125cc7044 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.392833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1d04-0370-4486-9e3b-958125cc7044 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.395814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 129/267] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:45.410453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d474-1b0a-4362-9a69-2a4f6c6e4e8d None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.410766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d474-1b0a-4362-9a69-2a4f6c6e4e8d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.411015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d474-1b0a-4362-9a69-2a4f6c6e4e8d None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.415157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 130/268] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:45.416394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.419163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.419465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.445206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:45.446706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:45.470947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.475957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 139/269] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:45.477680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.478273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.478653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.497679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:45.499106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:45.542618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 131/270] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:45.543497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.614372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 140/271] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:45.615315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.623217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.679590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df63d98-7946-483c-af7c-49d8a7fcd996 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:45.729672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:45.730377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.731085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.731365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.735342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:45.736806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:45.738571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 132/272] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:45.741640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.763632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b92db-656e-4ed6-8c05-bd8594c12724 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:45.800709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:45.801706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.802224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.802721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.810171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.815073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:45.820490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:45.821353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:45.822192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:45.824622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 141/273] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:45.825568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.882411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddc82-0c7c-4cea-ac78-6762d9a3ccd4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:45.934555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:45.939757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:45.940418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:45.940833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:45.941232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:45.942890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:45.945160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:45.947971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 133/274] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:45.949936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:45.961295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f284-c722-4e8d-8930-6c473d24ccec None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.022597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.023237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.023520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.024746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.025471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=a213d9870a7e49bbaf00cb4ef34cb273) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:46.028349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.028708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.037017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 142/275] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/role_assignments?user.id=a213d9870a7e49bbaf00cb4ef34cb273&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.038136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.076977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28924-3b21-4b98-b905-67d3383fb591 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.121902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.122552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.122835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.123113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.123741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.123911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:46.124202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.126263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd3dfd-24a0-4024-b9d9-a6b044c6345e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.137692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 134/276] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.139231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd508-81f7-48e7-80de-9ef4eb9a89f7 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.139524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd508-81f7-48e7-80de-9ef4eb9a89f7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.139801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd508-81f7-48e7-80de-9ef4eb9a89f7 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.145439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 135/277] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:46.146654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd508-81f7-48e7-80de-9ef4eb9a89f7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.191110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0110996-3578-466a-9729-d7af985e1140 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.224205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.233458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.235592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.235880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.236153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.237621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.238453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a73ae-c095-4fe1-a31d-b26592819e51 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.263136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 143/278] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:57:46 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:46.264676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.265614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.266120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.278420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd508-81f7-48e7-80de-9ef4eb9a89f7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.293432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:46.295317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:46.320420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.321191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.321523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.321805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] PATH_INFO: `/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.340337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] RBAC: Authorizing `identity:get_user(user_id=baf0949a1ddd4a5c889a39b1f728a5fe)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.341819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.344849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 136/279] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:46 2022] GET /identity/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.354678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.404866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 144/280] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:46.427997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.479448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a100-8000-4c9f-9e59-f30953962f20 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.482934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.505531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.526494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.527358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.527816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.528265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.529199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, user.id=baf0949a1ddd4a5c889a39b1f728a5fe, scope.system=all) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:46.529666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.532888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.542214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 137/281] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 28 08:57:46 2022] GET /identity/v3/role_assignments?user.id=baf0949a1ddd4a5c889a39b1f728a5fe&scope.system=all&role.id=6273caafd8e64714b134898490672923 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.573878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e765e-486e-4adc-8a37-9d3f34b097fe None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.620588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.621649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.622214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.622490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.635842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:46.641506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:46.642441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:46.643216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:46.645994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 145/282] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:46.649216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.771461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81b6d4-067b-4b33-9225-0a02979a0389 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:46.806698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:46.807676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:46.808159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:46.808632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:46.809604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:46.810142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:46.811729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:46.824998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 138/283] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:46.831385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:46.992918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:46.996532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719b56-fe00-4006-a6ee-74c67bb9a412 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:47.033265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:47.033859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.034149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.034429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] PATH_INFO: `/v3/users/acce5d4ebb8244f98da8b96243481f04` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.035649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] RBAC: Authorizing `identity:get_user(user_id=acce5d4ebb8244f98da8b96243481f04)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:47.037120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:47.038771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 146/284] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:46 2022] GET /identity/v3/users/acce5d4ebb8244f98da8b96243481f04 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:47.044334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:47.130294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:47.196803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175905e-8d2c-49a4-b9d3-ff6855fcddfa None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:47.232622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:47.233283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.233560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.233859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.235568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:47.237021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db356577-aff0-4ba6-9847-f14ea0fc94d4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:47.238740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 139/285] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:47 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:47.244790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:47.391876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dbffe-6e23-4605-8700-e92c1d29f72b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:47.433564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:47.434294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.434580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.434860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/acce5d4ebb8244f98da8b96243481f04/roles/a65333a7deba49b9988b74c07a81dd94` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.436873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, user_id=acce5d4ebb8244f98da8b96243481f04, role_id=a65333a7deba49b9988b74c07a81dd94)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:47.438034 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61762c-76bf-404b-8b0c-9266eff35328 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.438324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61762c-76bf-404b-8b0c-9266eff35328 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.438600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61762c-76bf-404b-8b0c-9266eff35328 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.438981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:47.439956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 140/286] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:47.445067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.445349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.445625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.455068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 147/287] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:57:47 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/users/acce5d4ebb8244f98da8b96243481f04/roles/a65333a7deba49b9988b74c07a81dd94 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:47.477418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:47.478893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:47.578108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 141/288] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:47.616456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:47.661703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:47.751606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17736e-2a73-4a00-b69b-a4dbfa01c8f6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:47.798720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:47.799376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:47.799656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:47.799929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:47.815835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:47.820824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:47.821684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:47.822501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 148/289] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:47.828828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:47.874302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:47.947498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cd00f-0007-4211-81fd-fb9053039d0a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.002755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.003438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.003716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.003988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.004712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:48.005013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.006542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.017675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 142/290] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:48.023720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.156309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:48.176825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fcf5-d5d9-4e47-8044-4c473a9c4908 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.215099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.215663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.215941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.216217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] PATH_INFO: `/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.217392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] RBAC: Authorizing `identity:get_user(user_id=5b48f004a9cc4cdda87f3004bed5f7e5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.219043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.221083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 149/291] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:48 2022] GET /identity/v3/users/5b48f004a9cc4cdda87f3004bed5f7e5 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:48.227238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.407441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff92ea-4316-4f56-bdee-f830a5c90d80 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.420091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:48.457642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.458305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.458585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.458862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.544494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.546318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.679037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 143/292] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:57:48.685665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.808102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883b37-b073-4553-b460-e55dd9bc5c8f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:48.860626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:48.861309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:48.861590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:48.861888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:48.866709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:48.868202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c1d49-8c09-4291-8722-6aed600cfc55 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:48.870146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 150/293] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:48 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:48.876195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:48.894618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.028451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e668-4039-4489-a08c-d9804dbde03c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.077770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.078852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.079131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.079402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.080188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=5b48f004a9cc4cdda87f3004bed5f7e5) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:49.080493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:49.081964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f93bf-1179-4e6d-9588-078aaeefa941 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:49.090938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 144/294] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:48 2022] GET /identity/v3/role_assignments?user.id=5b48f004a9cc4cdda87f3004bed5f7e5&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6273caafd8e64714b134898490672923 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:49.172029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ebf-0b00-4557-beba-be17e2d59821 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.172538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ebf-0b00-4557-beba-be17e2d59821 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.172611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8ebf-0b00-4557-beba-be17e2d59821 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.174187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 151/295] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:49.179340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-1c09-4d1c-83c1-c395eb1eca4c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.179612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-1c09-4d1c-83c1-c395eb1eca4c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.179886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-1c09-4d1c-83c1-c395eb1eca4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.208198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2e4c8c7b-1c09-4d1c-83c1-c395eb1eca4c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:49.209411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2e4c8c7b-1c09-4d1c-83c1-c395eb1eca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:49.293997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.311223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 145/296] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:49.320012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4fa24-ae89-42e9-a909-7d1e8fa3b48d None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.320559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4fa24-ae89-42e9-a909-7d1e8fa3b48d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.321049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4fa24-ae89-42e9-a909-7d1e8fa3b48d None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.324553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 152/297] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:49.327709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca721-bf29-4fa6-8953-d2ad13ba9ee4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.328597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca721-bf29-4fa6-8953-d2ad13ba9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.328830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca721-bf29-4fa6-8953-d2ad13ba9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.335972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4fa24-ae89-42e9-a909-7d1e8fa3b48d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.350366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b7cca721-bf29-4fa6-8953-d2ad13ba9ee4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:49.351660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b7cca721-bf29-4fa6-8953-d2ad13ba9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:49.364907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.432474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 146/298] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:49.433803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d26d-1142-408d-a867-18e05f48f9d9 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.434121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d26d-1142-408d-a867-18e05f48f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.434412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d26d-1142-408d-a867-18e05f48f9d9 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.445928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 147/299] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:49.447351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.447642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.447922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.475194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:49.476399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:49.480417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4fa24-ae89-42e9-a909-7d1e8fa3b48d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.520860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.521509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.521792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.522101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.535839 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:49.541356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:49.542311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:49.543076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:49.545988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 153/300] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:49.548104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.559686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 148/301] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:49.560639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.629650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.667654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.686687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafa87a-88e2-417e-85b9-a838f88f0c19 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.701854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260079-7ce8-4383-b804-61a76850b665 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.720374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.721081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.721368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.721663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.731934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:49.737923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:49.739600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 154/302] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:49.741199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.769478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.771080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.771941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.772709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.774635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:49.775467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:49.778693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:49.802973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 149/303] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:49.803199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.857089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56671faa-1683-4aac-bb5d-cfe42be03a01 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.897690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.898301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.898631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.898913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.900236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:49.901872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:49.908282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 155/304] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:49.908282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:49.923743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.937428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9b493-92df-4c3a-9170-009cec06f02a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:49.960394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:49.972889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:49.973506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:49.973785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:49.974094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:49.974913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:49.975322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:49.976825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:49.985762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 150/305] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:49.986969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.024824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14807e98-c017-4cce-9555-5fc431bfb336 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.058468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.059118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.059402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.059682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] PATH_INFO: `/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.060861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] RBAC: Authorizing `identity:get_user(user_id=baf0949a1ddd4a5c889a39b1f728a5fe)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.062415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47b241-94b8-411e-9185-c1c787bf10c3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.064030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 156/306] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:49 2022] GET /identity/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:50.069723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec61a8-41a3-4bcc-8dd0-83519f70e3c5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.070036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec61a8-41a3-4bcc-8dd0-83519f70e3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.070321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec61a8-41a3-4bcc-8dd0-83519f70e3c5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.071571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 157/307] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:50.072269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec61a8-41a3-4bcc-8dd0-83519f70e3c5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.086001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f904-287c-488a-9784-b51e803e6676 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.131170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.131800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.132083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.132359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] PATH_INFO: `/v3/groups` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.133020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.134508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.152046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:50.153726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 151/308] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Jul 28 08:57:49 2022] POST /identity/v3/groups => generated 229 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:57:50.154622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.182654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec61a8-41a3-4bcc-8dd0-83519f70e3c5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.221206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:50.221824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.222541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.222795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.223068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] PATH_INFO: `/v3/users/9b7c32238cde4efc8524354fa5610fd6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.240765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7c32238cde4efc8524354fa5610fd6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.242314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414d7e5-f7d2-4b4e-b86d-2a030abd39b5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.244082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 158/309] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/users/9b7c32238cde4efc8524354fa5610fd6 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:50.245298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.245579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.245865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.264554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:50.266341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:50.295330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688f9bd-15c8-4d63-89a9-43548191d732 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.339106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.340022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.340436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.340882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] PATH_INFO: `/v3/system/users/baf0949a1ddd4a5c889a39b1f728a5fe/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.342423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=baf0949a1ddd4a5c889a39b1f728a5fe, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.343916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.352159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 159/310] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:50.353185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.359495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 152/311] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Jul 28 08:57:50 2022] PUT /identity/v3/system/users/baf0949a1ddd4a5c889a39b1f728a5fe/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:57:50.381196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.457172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:50.505491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48e8bc-bd3e-4cb8-a3b1-b83783a7eff1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.510121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e304d-8b48-4dc2-8a52-0eeb8097f562 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.522273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:50.551214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.552083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.552498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.552917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.554012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, user.id=9b7c32238cde4efc8524354fa5610fd6, scope.system=all) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:50.554569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.556520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.557934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.558587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.558867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.559136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.563831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 160/312] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/role_assignments?user.id=9b7c32238cde4efc8524354fa5610fd6&scope.system=all&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:50.576307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:50.583005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:50.584135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:50.585096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:50.586918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 153/313] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:57:50.593905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.712718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a0efa-8363-4361-847b-11bd35bb6bb8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.744357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:50.751785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.752645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.753073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.753493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.754410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:50.754929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.756602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.766192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 161/314] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:50.774702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:50.881970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373219-991f-4445-95ef-60fd9a389987 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:50.917006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:50.917667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:50.917982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:50.918264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] PATH_INFO: `/v3/users/acce5d4ebb8244f98da8b96243481f04` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:50.919523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] RBAC: Authorizing `identity:get_user(user_id=acce5d4ebb8244f98da8b96243481f04)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:50.920987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:50.929873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 154/315] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/users/acce5d4ebb8244f98da8b96243481f04 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:50.930016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.047435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165ff8c-a636-438f-a51b-a61d62da983f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.082535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.083171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.083450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.083728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.150522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.165266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.167067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.272640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 162/316] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:57:51.278211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.321828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.395215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68543-adc1-4a99-a653-56366b7c0d90 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.432164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.432818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.433100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.433377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.434561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.436016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8de1a-a71f-4e7f-a3b4-035e6fab9f05 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.438058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 155/317] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:51.446991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:51.539751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:51.549094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db015de2-947e-48a5-842a-b0916a2cab64 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:51.584722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:51.585266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:51.585548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:51.585824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:51.586650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, scope.project.id=c46131bba4b1483ba24d73c34490f88b, user.id=acce5d4ebb8244f98da8b96243481f04) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:51.586963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:51.588408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0b6e0-e771-44fc-a373-5f938354893a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:51.598994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 163/318] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:57:51 2022] GET /identity/v3/role_assignments?user.id=acce5d4ebb8244f98da8b96243481f04&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:52.326877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681429e-6920-497c-9ad8-38afc3243bec None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.327391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681429e-6920-497c-9ad8-38afc3243bec None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.327493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681429e-6920-497c-9ad8-38afc3243bec None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.329521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 156/319] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:52.334470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a9153-e11f-45e0-b5e3-3275b139b306 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.334859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a9153-e11f-45e0-b5e3-3275b139b306 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.335015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a9153-e11f-45e0-b5e3-3275b139b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.358116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a62a9153-e11f-45e0-b5e3-3275b139b306 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:52.359744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a62a9153-e11f-45e0-b5e3-3275b139b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:52.432207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 164/320] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:52.454599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681429e-6920-497c-9ad8-38afc3243bec None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.529151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8d91-b027-4ad5-ac0d-5dd83ff609f2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.529477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8d91-b027-4ad5-ac0d-5dd83ff609f2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.529698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8d91-b027-4ad5-ac0d-5dd83ff609f2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.533224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 165/321] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:52.538667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b7b2-b9e5-48cd-b019-a6bd2ac6b646 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.538950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b7b2-b9e5-48cd-b019-a6bd2ac6b646 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.539219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b7b2-b9e5-48cd-b019-a6bd2ac6b646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.557489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-efe9b7b2-b9e5-48cd-b019-a6bd2ac6b646 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:52.558684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-efe9b7b2-b9e5-48cd-b019-a6bd2ac6b646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:52.566429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681429e-6920-497c-9ad8-38afc3243bec None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.569935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:52.621654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.622266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.622554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.622833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.624003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:52.626105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:52.627708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 157/322] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:52.633084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.646834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 166/323] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:52.651485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04859a3-e342-4e27-a338-6fd8d35aee35 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.651862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04859a3-e342-4e27-a338-6fd8d35aee35 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.652198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04859a3-e342-4e27-a338-6fd8d35aee35 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.653797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 167/324] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:57:52.658829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.659014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.659295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.677705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:52.678932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:52.691261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:52.742353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891b643-0272-4303-890b-d89164fb8dcf None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.756863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 168/325] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:57:52.758016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.782430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.783049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.783325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.783596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] PATH_INFO: `/v3/groups` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.784250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:52.785684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:52.801076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 158/326] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Jul 28 08:57:52 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:57:52.802234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.847787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:52.872744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd16e2d-8384-45c2-bd0b-d1204071a6ca None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.914361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.915144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.915420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:52.915691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:52.935627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:52.940937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:52.942168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:52.948101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:52.948684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0974-aa4f-4379-ad3f-84fc18de1b1a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:52.948763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 169/327] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:52.948763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:52.958433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:52.998416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:52.999414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:52.999871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.000299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.024338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:53.030296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:53.031755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:53.032550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 159/328] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:53.038981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:53.097886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e953d9-3dfc-4445-820f-1b60b7dc38a9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.139838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.140753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.141032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.141326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.147180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:53.147180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.147180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.153080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 170/329] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:53.159978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:53.161652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:53.177177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddbbd2-91b1-4a6c-a499-9184a4b7e041 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.228286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.228918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.229199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.229473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.230226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:53.230534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.231941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.244797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 160/330] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:53.250932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:53.291489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:53.342272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dc94-fac1-4384-8023-d811fc0dac73 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.397085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.398074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.398182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.398538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] PATH_INFO: `/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.401950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] RBAC: Authorizing `identity:get_user(user_id=baf0949a1ddd4a5c889a39b1f728a5fe)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.403696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.406395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 171/331] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:53 2022] GET /identity/v3/users/baf0949a1ddd4a5c889a39b1f728a5fe => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:53.414743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:53.440360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd690f-7522-44bd-9d4e-c36bc011025c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.492955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.493710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.494270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.494556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] PATH_INFO: `/v3/users/9b7c32238cde4efc8524354fa5610fd6` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.496573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] RBAC: Authorizing `identity:get_user(user_id=9b7c32238cde4efc8524354fa5610fd6)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.498623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.501710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 161/332] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:53 2022] GET /identity/v3/users/9b7c32238cde4efc8524354fa5610fd6 => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:53.507099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:53.542868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:53.560537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43ecf-b370-41a9-a5af-f59f00d3f7bd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.610609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.611605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.612093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.612570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.614583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] RBAC: Adding query filter params (role.id=6273caafd8e64714b134898490672923, user.id=baf0949a1ddd4a5c889a39b1f728a5fe, scope.system=all) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:53.615078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.616569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b6572-89e5-4a86-9978-4ad7e1dbe8ce None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.622555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:53.637329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 172/333] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 28 08:57:53 2022] GET /identity/v3/role_assignments?user.id=baf0949a1ddd4a5c889a39b1f728a5fe&scope.system=all&role.id=6273caafd8e64714b134898490672923 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:53.654953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62512d28-7b95-4e6a-982a-0023e8bb516a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:53.724600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:53.725763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:53.726339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:53.726857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] PATH_INFO: `/v3/system/users/9b7c32238cde4efc8524354fa5610fd6/roles/a65333a7deba49b9988b74c07a81dd94` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:53.729576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9b7c32238cde4efc8524354fa5610fd6, role_id=a65333a7deba49b9988b74c07a81dd94)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:53.732586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c8a45-cb48-489d-820d-84ca8d188778 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:53.749998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 162/334] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Jul 28 08:57:53 2022] PUT /identity/v3/system/users/9b7c32238cde4efc8524354fa5610fd6/roles/a65333a7deba49b9988b74c07a81dd94 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:57:55.249346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d174d3-52a4-4e1b-9359-37a0d5ef55a9 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.250434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d174d3-52a4-4e1b-9359-37a0d5ef55a9 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.250434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d174d3-52a4-4e1b-9359-37a0d5ef55a9 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.251922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 173/335] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:55.258394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.258868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.258969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.287410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:55.288954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:55.375471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 163/336] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:55.410705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d174d3-52a4-4e1b-9359-37a0d5ef55a9 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:55.550830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d174d3-52a4-4e1b-9359-37a0d5ef55a9 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:55.593875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:55.594539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.594824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.595096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.606116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:55.612776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:55.613637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:55.614458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 174/337] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:55.621610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:55.649680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:55.763461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4871-b51c-4fda-af4f-edc0af15dab8 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:55.814638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:55.815205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.815490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.815767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.816504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:55.816809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:55.818243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818619e-f8a6-4e6c-9d03-45481925c5ed None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:55.827511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 164/338] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:55.833262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:55.862680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce4df9-7e58-4289-8729-df9a86e531a0 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.862998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce4df9-7e58-4289-8729-df9a86e531a0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.863244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce4df9-7e58-4289-8729-df9a86e531a0 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.864870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 165/339] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:55.870072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:55.870351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:55.870628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:55.902904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:55.904283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:55.915966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:55.995009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d2880-01c8-4672-b15c-ab8c8113114f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.046182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.047482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.047765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.048040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.096241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 166/340] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:56.135371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.152994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:56.154516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba722cd0-a14c-4d40-8e6f-b17610106a04 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:56.157169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.228140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.300369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a656a0b-44b2-4136-bb5b-fcbdf2cb2eb0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.308246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 175/341] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:56.310745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e285-8347-4afa-abda-1633a75cb627 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.314337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e285-8347-4afa-abda-1633a75cb627 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.314623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e285-8347-4afa-abda-1633a75cb627 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.316391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 176/342] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:56.320403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80e285-8347-4afa-abda-1633a75cb627 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.376404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.378126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.378416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.378721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.401910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-24f608c4-443c-4b6b-94b9-865751d2e016 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:56.406975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:56.407853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:56.408624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:56.410045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 167/343] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:56.413941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.414235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.414506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.445377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:56.447115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:56.478545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80e285-8347-4afa-abda-1633a75cb627 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.564534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.565604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.566163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.566663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.571307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:56.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:56.575953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 177/344] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:56.577128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.578350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.583714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 168/345] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:56.585710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.633828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.721157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d539-616f-4b03-a646-9697fc7e5c77 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.742903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec81e6-9521-4055-8fe6-b7c566c05812 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.762066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.762703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.762979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.763251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.763975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:56.764714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:56.766414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:56.779117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 178/346] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:56.779960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.783170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.783913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.784244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.784521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.791186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:56.792725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840deb0-12ae-4d1f-b4d6-d27ce667e186 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:56.794802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 169/347] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:56.801439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d4154-03e9-4a94-932e-f5b6f5d3ee69 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.801726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d4154-03e9-4a94-932e-f5b6f5d3ee69 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.802035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d4154-03e9-4a94-932e-f5b6f5d3ee69 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.803450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 170/348] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:56.804166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d4154-03e9-4a94-932e-f5b6f5d3ee69 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:56.893688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.896622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:56.924442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e37b-7195-4f12-9cf5-1b379b4bd9de None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.953798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d4154-03e9-4a94-932e-f5b6f5d3ee69 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:56.970670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.972271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.972882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.973426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.988070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:56.995684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:56.996452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:56.996832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:56.997205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:56.997358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:56.998412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 08:57:56.999387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:57.001504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 179/349] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:57.002464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.088599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.091219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.136583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc55ce9-1223-4a3a-82a6-c3c3fc9ab241 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.159287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.178863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.179808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.180489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.180899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.181993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=6273caafd8e64714b134898490672923, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:57.182534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.184212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5d3c3-c8d1-4f64-96a0-334bfcd44cd7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.190983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 180/350] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6273caafd8e64714b134898490672923 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.192506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.193040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.193527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.214236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 171/351] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:57.215108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.216177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:57.217672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:57.220819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.298414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 181/352] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:57.299779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.352273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0d61-a1d7-4438-9abb-69c5e2347a61 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.396099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.396782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.397062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.397343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.398121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:57.398426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.399877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.410746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 172/353] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.411407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.420869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb83ea-99ac-461d-97ee-b7c29d1a703b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.462236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.464462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.465423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.465959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.466445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.468170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.469965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.472006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 182/354] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.473043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.484572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.528863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcace8f-51fe-477e-bab3-82d1d9960acb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.571972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.573921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.574466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.574945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.590715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-6d3cdc53-6a20-4112-b899-c50eacdad467 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:57.598076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:57.598971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:57:57.599732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:57.602798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 173/355] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:57.605104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb8aa-7f04-4535-a417-0ccc423989ab None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.605541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb8aa-7f04-4535-a417-0ccc423989ab None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.606000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb8aa-7f04-4535-a417-0ccc423989ab None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.608193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 174/356] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:57.609349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb8aa-7f04-4535-a417-0ccc423989ab None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.610879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4410-13f5-4b10-9890-89f421d82262 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.654714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.655580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.655987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.656384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] PATH_INFO: `/v3/users/9b7c32238cde4efc8524354fa5610fd6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.657752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7c32238cde4efc8524354fa5610fd6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.659337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.661205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 183/357] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/users/9b7c32238cde4efc8524354fa5610fd6 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.662025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.734428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.762023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb8aa-7f04-4535-a417-0ccc423989ab None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.764328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:57.795382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5560909-3d93-413c-b411-a3c8ff254a20 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:57.800405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.801066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.801350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.801640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.802820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.804285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f7825-c7c9-4efe-8a42-7385e3ad5346 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.806490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 175/358] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.807658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff548b-2f96-451c-9320-b77b97fed08e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.807943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff548b-2f96-451c-9320-b77b97fed08e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.808223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff548b-2f96-451c-9320-b77b97fed08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.829254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0bff548b-2f96-451c-9320-b77b97fed08e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:57.830494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0bff548b-2f96-451c-9320-b77b97fed08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:57.845139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:57.845773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.846088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.846366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.847099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:57.847409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:57.848803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:57.857912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 184/359] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:57.858748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.931644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 176/360] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:57.932700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a7b3-0ad1-49b5-baed-83a9815ad65a None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:57.932987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a7b3-0ad1-49b5-baed-83a9815ad65a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:57.933264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a7b3-0ad1-49b5-baed-83a9815ad65a None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:57.934557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 177/361] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:57.935309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2a7b3-0ad1-49b5-baed-83a9815ad65a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:57.990768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9f9e4-2ec8-482d-85d8-6b464d4a8565 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.039429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.040053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.040336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.040609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.041412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] RBAC: Adding query filter params (role.id=a65333a7deba49b9988b74c07a81dd94, user.id=9b7c32238cde4efc8524354fa5610fd6, scope.system=all) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:58.041723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.045620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.047941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.055936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.064626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 185/362] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/role_assignments?user.id=9b7c32238cde4efc8524354fa5610fd6&scope.system=all&role.id=a65333a7deba49b9988b74c07a81dd94 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:58.066502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.083512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2a7b3-0ad1-49b5-baed-83a9815ad65a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.127399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.128093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.128380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.128664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.129551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:58.129893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.131503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23359f8e-b3a0-4b4e-900d-2f1b4ca4fd92 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.141875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 178/363] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:57:57 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:58.141875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.141875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.141875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.175149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:58.176411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:58.206897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1806a5-dca7-437f-8697-43455a8d63dd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.255596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.257262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.257754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.353647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 179/364] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:57:58.354736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.373240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.384484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.386569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.473351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.505294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24d6ef-ee59-487f-9848-3c86a88fbee4 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.540765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 186/365] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:58.543398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.552084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.552742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.553015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.553291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.567383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:58.573479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:58.575308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:57:58.576938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:57:58.580109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 180/366] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:58.580975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.719055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d72c5-6b45-4337-84bb-8b9e4cf80835 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.737315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.748036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896cc1-baff-4e9e-8259-c0d33e866ea2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.783068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.784430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.784924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.785671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.790710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.792201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:58.794268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 181/367] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:58.795059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.798571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.799211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.802751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.802751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.805049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:58.822127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:57:58.828618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:57:58.829547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:57:58.830416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:57:58.832936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 187/368] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:57:58.834063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:58.950853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987117b1-6dd4-4efc-af43-c4e70528532c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.989503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a453b0-a4eb-4045-a4f6-80cbe80d95c8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:58.990735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:58.991475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:58.991865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:58.992152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:58.992883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:58.993190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:58.995740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.006277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 182/369] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:59.007117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.037947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.038611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.038894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.039172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.040520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.042866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.045421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 188/370] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:57:58 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:59.047077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.083405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.102465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.154882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bccf3-4f07-48d0-865f-7261a668063e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.197226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01365425-6ef3-4c8f-bae7-fadc767b80b0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.208684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.209344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.209629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.209934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.210680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:59.211444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.213248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.224891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 183/371] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:59.226693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.243963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.244523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.244850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.245129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.324235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.325726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.353453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.364258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033d5dd-6a53-4e7e-9cd1-00e43e47c71e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.407244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.424506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.425285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.425587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.425897 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.427252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=6273caafd8e64714b134898490672923, scope.project.id=c46131bba4b1483ba24d73c34490f88b) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:57:59.427682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.429327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.438135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 184/372] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6273caafd8e64714b134898490672923 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:59.439082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.451463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 189/373] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:59.457227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.622986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59315-470b-45aa-a136-d9c6a42eb5ea None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.672822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad610a-de51-4ccd-800e-e6c7edbd2353 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.682661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.682874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.683103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.683296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.702729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.727374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.728687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.743147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.744103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.746012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f89557-0712-489a-85f1-797b2957745b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.748244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 190/374] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:57:59.753702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d91a2c-5668-41fc-a34a-f193dde5ced1 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.754035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d91a2c-5668-41fc-a34a-f193dde5ced1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.754314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d91a2c-5668-41fc-a34a-f193dde5ced1 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.754618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:57:59.755850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 191/375] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:57:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:57:59.756663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d91a2c-5668-41fc-a34a-f193dde5ced1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:57:59.762490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.765252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a52bb-a4a2-42dd-b332-a1035c24f2d4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.886940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d91a2c-5668-41fc-a34a-f193dde5ced1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:57:59.900305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 185/376] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:57:59.901993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.902619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.902905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.931788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:57:59.933120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:57:59.943512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:57:59.944570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:57:59.945074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:57:59.945598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:57:59.947348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:57:59.949056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:57:59.951272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 192/377] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:57:59.952299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.009120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 186/378] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:00.010351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.032289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.050236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.084046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2528487-594f-4838-8ce8-32f4244fc829 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.119841 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.120689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.121070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.121354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] PATH_INFO: `/v3/projects/66b6795243644c5d8ae6b99e815d8914` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.122590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] RBAC: Authorizing `identity:get_project(project_id=66b6795243644c5d8ae6b99e815d8914)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.124046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.126082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 193/379] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:57:59 2022] GET /identity/v3/projects/66b6795243644c5d8ae6b99e815d8914 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:00.126955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.139568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb46113-4e67-4c56-925b-8214a5e1379a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.181335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.182143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.182430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.182763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.183571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=c46131bba4b1483ba24d73c34490f88b) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:00.183883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.185460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.192155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 187/380] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:00.193002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.307068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7c9c-8adb-4ef5-a7c7-8b555bab539b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.310261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.319658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c61644-676b-4902-9bad-d0a114d6c053 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.362371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.363091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.363379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.363657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] PATH_INFO: `/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.368008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.370261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] RBAC: Authorizing `identity:get_group(group_id=c0c7f1259348472ca5b3ca4095c38b7f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.371763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.373873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 188/381] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:00.380219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.405732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.406405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.406687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.406958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.433226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:00.445477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:00.446482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:00.447272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 194/382] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:00.459765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.551029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d99e5-0e65-46b1-8f19-56d494ff7666 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.611965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.612605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.612884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.613164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.614199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] RBAC: Adding query filter params (group.id=c0c7f1259348472ca5b3ca4095c38b7f, role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=66b6795243644c5d8ae6b99e815d8914) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:00.614514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.615967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.625536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457467a-8fce-48e8-8b70-67916c944bd2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.629575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 189/383] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/role_assignments?group.id=c0c7f1259348472ca5b3ca4095c38b7f&scope.project.id=66b6795243644c5d8ae6b99e815d8914&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:00.674367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.675022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.675181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.675459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.676208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] RBAC: Adding query filter params (name=member) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:00.676582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.678125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.689966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 195/384] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:00.698344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.716053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.838542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81eea5-73c4-4cd7-80e1-9402b0e19775 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:00.880690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:00.881370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.881651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.881941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.884035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:00.886439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc4768-6c38-436f-bc52-ee1f307d0dc4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:00.891983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 190/385] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:00.897801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:00.973243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:00.995000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93cfb8-67b4-4284-9c1f-fa39981eb279 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:00.995334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93cfb8-67b4-4284-9c1f-fa39981eb279 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:00.995549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93cfb8-67b4-4284-9c1f-fa39981eb279 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:00.996915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 191/386] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:01.003943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.004204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.004491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.033059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:01.035541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:01.055605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a236cba-b94e-4519-997b-bce16aa2594c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.136674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 192/387] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:01.159189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.159856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.160172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.160448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.164682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.164682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.165059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.166515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 196/388] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:00 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:01.178061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.185872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.313366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9e47-5114-44cc-99e4-a2d11d081b03 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.332632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.336864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315261-5f1f-4b78-b81f-30a71e1bef8c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.356554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.357555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.358215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.360105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/groups/9e11188f329f4f03971d4f6a8343c73d/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.363094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, group_id=9e11188f329f4f03971d4f6a8343c73d, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.365480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.388366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 197/389] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 28 08:58:01 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/groups/9e11188f329f4f03971d4f6a8343c73d/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:01.401210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.401904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.402231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.402509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.422552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:01.436552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:01.438073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:01.438939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 193/390] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:01.445959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.603702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac6bd-546f-456b-be22-8b378e695d40 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.643622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.644255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.644539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.644816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.645547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:01.645877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.647263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.656938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 198/391] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:01.662314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.743965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.787817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954c006-40b4-40f3-8bff-4dfa28b2592e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:01.834431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:01.835093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:01.835377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:01.835657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:01.836888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:01.838392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:01.841030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 194/392] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:01.846219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:01.891571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:01.989534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70255e67-9fef-4aaa-ae58-bb7fa3e19f02 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:02.027311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:02.027931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.028219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.028498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.029721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:02.031747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0aaa31-9b52-40be-b792-5189412dd96d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:02.033787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 199/393] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:01 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:02.042745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:02.168828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf184e-1136-4ce0-8a91-5af4d546c553 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:02.217431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:02.218177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.219052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.219327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/groups/9e11188f329f4f03971d4f6a8343c73d/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.226705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85a9aedd0d6451f83021f1ff0323ac8, group_id=9e11188f329f4f03971d4f6a8343c73d, role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:02.228171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c74723-9484-4315-a92b-d65896085f26 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:02.253791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:02.259095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 195/394] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 28 08:58:02 2022] PUT /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8/groups/9e11188f329f4f03971d4f6a8343c73d/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:02.263240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd814-e173-4d6f-ba97-26945128be0f None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.263530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd814-e173-4d6f-ba97-26945128be0f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.264066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd814-e173-4d6f-ba97-26945128be0f None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.267581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 200/395] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:02.273312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.273621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.273877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.298674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:02.299924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:02.380665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 196/396] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:02.403941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fd814-e173-4d6f-ba97-26945128be0f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:02.576945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fd814-e173-4d6f-ba97-26945128be0f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:02.593809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:02.649965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:02.651079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.651576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.652075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.676846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:02.687515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:02.688419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:02.689230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:02.690343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 201/397] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:02.698623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:02.844121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc28eab-150f-4641-b9f3-16971bb5616d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:02.884745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:02.899531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:02.900511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.900990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.901513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.902491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:02.902995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:02.904587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a972c71-df28-409c-9d28-c6ee7d910f7c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:02.915084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 197/398] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:58:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:02.931658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:02.982750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb829f-57b3-4e29-b9d9-e6f102e7b3f5 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.983430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb829f-57b3-4e29-b9d9-e6f102e7b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.984051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb829f-57b3-4e29-b9d9-e6f102e7b3f5 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:02.986200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 198/399] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:02.992993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:02.993588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:02.994098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.032832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:03.034137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:03.087627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0ce0-0974-4648-9a20-51d0cd5049ee None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.136968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.138073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.138555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.139038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.141833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.143510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.145692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 202/400] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:02 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:03.151407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.169866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 199/401] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:03.207428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:03.234361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.286008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbeaf7-dcdb-48dd-9853-d9ce30f3cdf6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.345120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.345959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.346335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.346679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.348159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.349936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.352203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 203/402] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:03.363200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.414236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd3e0e-431b-44ae-91ca-ee9654229798 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.461932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:03.484170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.484881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.485172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.485432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.496040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c0300f48-b0b4-4767-88bc-f0503356fb5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:03.501512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:03.502441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:03.503219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 200/403] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:03.504400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20254-4e9d-43a1-8617-c4d85fd26e93 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.504400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20254-4e9d-43a1-8617-c4d85fd26e93 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.504546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20254-4e9d-43a1-8617-c4d85fd26e93 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.506032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 201/404] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:03.510789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20254-4e9d-43a1-8617-c4d85fd26e93 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.539675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f3c6-0b22-47e5-aac8-9398c4a823b7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.541937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:03.587069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.588077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.588366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.588649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/groups/9e11188f329f4f03971d4f6a8343c73d/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.591802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, group_id=9e11188f329f4f03971d4f6a8343c73d, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.593293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0231ea-94f7-4f5a-b549-e9d9729cb13d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.613259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 204/405] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 28 08:58:03 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/groups/9e11188f329f4f03971d4f6a8343c73d/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:03.615585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.616043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.616329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.625864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20254-4e9d-43a1-8617-c4d85fd26e93 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.644029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:03.645332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:03.687689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.688548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.688836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.689169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.689945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:03.690260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.692296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.703217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 202/406] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:03.709156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.769650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:03.808561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 205/407] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:03.834629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.834815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96781b-9f28-403a-ad4b-21ed5dfaf3df None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:03.884470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:03.885467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:03.886031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:03.886527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:03.887925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:03.889670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:03.891582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 203/408] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:03.898206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:03.958186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:03.971409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138e178-5f82-4328-b102-b6b586b50ddf None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.017919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.018883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.019526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.019795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.025058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eb392-5f9c-49f8-9dd1-3f3e24025290 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.035700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:04.042801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:04.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:04.044489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:04.045099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 206/409] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:04.050206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:04.052796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.075476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.076183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.076465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.076793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.086663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.088127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.090169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 204/410] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:03 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:04.098764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.164042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc50322-02d5-4ea0-9827-b144f1a31ac5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.234379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.234951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.235228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.235493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.236379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:04.236680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.238871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9bb2d-a70c-42d9-9b36-32a4f35de72f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.249628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 207/411] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:04.251269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be13a01-46b7-40ec-8f55-2b6070f9a9c5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.251835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be13a01-46b7-40ec-8f55-2b6070f9a9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.252111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be13a01-46b7-40ec-8f55-2b6070f9a9c5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.257540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 208/412] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:04.258408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be13a01-46b7-40ec-8f55-2b6070f9a9c5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.275234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:04.277438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba414b-cf5b-4edc-8473-1294ec9b6748 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.324914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.325896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.326192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.326477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b/groups/9e11188f329f4f03971d4f6a8343c73d/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.329673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] RBAC: Authorizing `identity:create_grant(project_id=c46131bba4b1483ba24d73c34490f88b, group_id=9e11188f329f4f03971d4f6a8343c73d, role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.331433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9c92a-92a1-43ac-b076-30cf59e6d91e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.351578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 205/413] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 28 08:58:04 2022] PUT /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b/groups/9e11188f329f4f03971d4f6a8343c73d/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:04.353664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.353987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.354327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.372021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be13a01-46b7-40ec-8f55-2b6070f9a9c5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.380338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:04.381625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:04.442354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:04.456094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.456826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.457110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.457403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] PATH_INFO: `/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.462752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] RBAC: Authorizing `identity:get_group(group_id=c0c7f1259348472ca5b3ca4095c38b7f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.464290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.470687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 209/414] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:04.472813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 206/415] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:04.477043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.519660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.596200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:04.618351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2171cf-e235-4117-a887-e4064bac17bf None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.651045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d43be-6fb7-4f6b-b472-d4a056b58d2b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.657872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.657872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.657872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.657872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] PATH_INFO: `/v3/projects/66b6795243644c5d8ae6b99e815d8914` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.658214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] RBAC: Authorizing `identity:get_project(project_id=66b6795243644c5d8ae6b99e815d8914)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.658646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.660789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 210/416] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/projects/66b6795243644c5d8ae6b99e815d8914 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:04.675867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.699532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.700577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.701057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.701575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.727853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:04.735438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:04.736302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:04.737122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:04.742318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 207/417] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:04.750156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:04.787858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:04.862837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7c976-6154-4847-b75d-5b8dfff8cb48 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.940949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fffcf3-670c-4327-be6f-74765a85dd8b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:04.954566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:04.955661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:04.956141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:04.956660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:04.957586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:04.958136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:04.961959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:04.974809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 208/418] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:04.980705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.003166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.004967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.005927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.006436 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] PATH_INFO: `/v3/projects/66b6795243644c5d8ae6b99e815d8914/groups/c0c7f1259348472ca5b3ca4095c38b7f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.008998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=66b6795243644c5d8ae6b99e815d8914, group_id=c0c7f1259348472ca5b3ca4095c38b7f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:05.011724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06ddcf-c060-477a-86e9-ddf13ed018c5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:05.047773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 211/419] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Jul 28 08:58:04 2022] PUT /identity/v3/projects/66b6795243644c5d8ae6b99e815d8914/groups/c0c7f1259348472ca5b3ca4095c38b7f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:05.125556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:05.135661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d437-59fc-4afa-b475-c2a7e06b9025 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:05.175814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.176455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.176737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.177013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.229966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b47f3-a3f6-4a6e-9ca6-224b1b22fa50 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.230293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b47f3-a3f6-4a6e-9ca6-224b1b22fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.230549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b47f3-a3f6-4a6e-9ca6-224b1b22fa50 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.233302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 212/420] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:05.238900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.239323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.242964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.263463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:05.265023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:05.266154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:05.267775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:05.274761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:05.359717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 213/421] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:05.385863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.396113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:05.414295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 209/422] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:05.423841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.558014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08f6c3-0470-480b-b424-ab6d7180d73f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:05.558242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e3b2f-0b97-4c5e-8234-6c56de37004c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:05.593981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.594624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.594901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.595171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.599033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:05.600481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:05.603674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 210/423] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:05.610768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.633280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.633947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.634226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.634495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.656850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:05.659003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:05.664522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:05.665603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:05.666522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:05.670006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 214/424] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:05.674304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.731814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:05.816829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd6c81-2129-490e-bd7b-a2322d1247b2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:05.863822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae621b7-cd5a-4a77-8789-f6457a61af74 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:05.878049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.878705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.878982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.879254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.881581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:05.883055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:05.885800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 211/425] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:05.894425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:05.931066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:05.931693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:05.931969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:05.932239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:05.932960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:05.933262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:05.934671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:05.956491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 215/426] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:05.957435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.050677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8663880-6fff-4d95-bf5e-cc086b5b5d58 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.086496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.087172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.087455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.087725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.088507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=6273caafd8e64714b134898490672923, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:06.088814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.090321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4c6f8-cbd3-4b93-8ab1-fd6d755ce564 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.097764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 212/427] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=6273caafd8e64714b134898490672923 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:06.098244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.111865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.169299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae2a6-305e-4d60-a2a1-c01b67eaf7d5 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.169588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae2a6-305e-4d60-a2a1-c01b67eaf7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.169775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae2a6-305e-4d60-a2a1-c01b67eaf7d5 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.172356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 213/428] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:06.179663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.179970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.180222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.181891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7d1da-ff45-4188-94a9-592e8432b3ab None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.208960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:06.210385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:06.259300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.259945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.260219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.260486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.330153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 214/429] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:06.358295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.374862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.376058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.462774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.512567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af3304-0967-4330-958f-a4411116dc48 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.517404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 216/430] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:06.526451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.543992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.561438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.562795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.563253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.563939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.581332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:06.589178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:06.590307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 08:58:06.591283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:06.594536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 215/431] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:06.599894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.664252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edaf1a-c607-4e72-b12c-ea48c3f67a94 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.706383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.707121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.707466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.708208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] PATH_INFO: `/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.709060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] RBAC: Authorizing `identity:get_project(project_id=a85a9aedd0d6451f83021f1ff0323ac8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.710881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.713108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 217/432] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/projects/a85a9aedd0d6451f83021f1ff0323ac8 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:06.721564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.738019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c423-f62f-42d2-bd2d-f68b7e021841 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.756200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.780554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.781731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.782043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.782322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.783041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:06.783348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.784731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255179-bb14-4ef7-8278-ff9da4afadfa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.793587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 216/433] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:06.794981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff85c-9de2-4fbf-9c94-50d643932e07 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.795274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff85c-9de2-4fbf-9c94-50d643932e07 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.795602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff85c-9de2-4fbf-9c94-50d643932e07 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.800995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 217/434] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:06.801826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ff85c-9de2-4fbf-9c94-50d643932e07 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:06.851338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5dd05-6e12-4c73-a90d-333426a58da7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.861146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:06.896296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.896960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.897238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.897510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.898736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:06.900176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ad970-b9ea-476c-9cbb-6b98d11d7dd3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:06.901984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 218/435] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:06.903034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.903261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.903504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:06.933750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:06.934736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ff85c-9de2-4fbf-9c94-50d643932e07 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:06.935847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:06.973315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:06.974097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:06.974376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:06.974637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.030058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 219/436] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:07.031321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.064564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.066144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.068150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.116351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.151578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200aa359-a035-4ef9-9925-e4c7c11bd6e1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.174092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 218/437] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:07.175252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.194529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.195159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.195439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.195720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.196520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:07.196829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.198311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.213864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 220/438] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=a85a9aedd0d6451f83021f1ff0323ac8&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:07.213864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.307421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa32f95-1849-4d1f-a3b2-a4552b72ecf4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.339912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.350058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.350854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.351203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.351307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5acb3-9230-4d96-8128-84e843b740e0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.351597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.367921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:07.373627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:07.375323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:07.376095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:07.376809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 219/439] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:07.382336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.387257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.404766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.405692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.406294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.406708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.408299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.410353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.412584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 221/440] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:07.419059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.520267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba16dc-c091-4065-95cf-12ff9f9f1de4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.551864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce477b4-530b-44c7-a6fe-b826c0b0eb91 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.566421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.566981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.567265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.567542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.568275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:07.568586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.570017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.579397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 220/441] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:07.585018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.596645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.597534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.598574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.598574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.601028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.602603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba06a8-c11b-4825-bb6e-2f584d123511 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.604477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 222/442] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:07.610856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3213-9c3a-4c4a-89f7-6d34dc0185e1 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.611145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3213-9c3a-4c4a-89f7-6d34dc0185e1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.611425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3213-9c3a-4c4a-89f7-6d34dc0185e1 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.612754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 223/443] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:07.613501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3213-9c3a-4c4a-89f7-6d34dc0185e1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.648848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.667257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.706748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721a15-71dc-4b39-85cb-1f846266fa87 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.729587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3213-9c3a-4c4a-89f7-6d34dc0185e1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:07.741456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.742097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.742377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.742649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.763268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:07.763895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.764180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.764456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.765295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=6273caafd8e64714b134898490672923, scope.project.id=c46131bba4b1483ba24d73c34490f88b) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:07.765644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.767240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea809b9-7239-456b-b51a-273b29320a93 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.773348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 224/444] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=6273caafd8e64714b134898490672923 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:07.774568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:07.774856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:07.775174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:07.798423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:07.799699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:07.847908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:07.849328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:07.878594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 225/445] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:07.902017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:07.923757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.935229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:07.969540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 221/446] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:07.974612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.031939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9530e3-5063-4424-9c23-3cb0def5fe42 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.073350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.074096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.074379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.074652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.088413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:08.091048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e7ad8-43d3-4814-8fc9-e1554e66cd4a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:08.095345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:08.096329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:08.097122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 226/447] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:08.103728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.133698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.134414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.134698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.134988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] PATH_INFO: `/v3/projects/c46131bba4b1483ba24d73c34490f88b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.136498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] RBAC: Authorizing `identity:get_project(project_id=c46131bba4b1483ba24d73c34490f88b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.137867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.140994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 222/448] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:07 2022] GET /identity/v3/projects/c46131bba4b1483ba24d73c34490f88b => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:08.145558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.181933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.197375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.223737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c269-b2b7-4032-9ae5-de7df0cf11c2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.254963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6bca1-8fd5-4118-8037-ad7a32994369 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.258285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.258897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.259174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.259443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.260151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:08.260452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.261828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.270405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 227/449] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:08.275845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.290035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.290688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.290968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.291246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] PATH_INFO: `/v3/groups/9e11188f329f4f03971d4f6a8343c73d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.292398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] RBAC: Authorizing `identity:get_group(group_id=9e11188f329f4f03971d4f6a8343c73d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.293863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.295462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 223/450] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/groups/9e11188f329f4f03971d4f6a8343c73d => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:08.301162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.372793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c07e4-dfa8-4f6c-85aa-0a93a27a5948 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.398682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b921db-2c6c-45dd-b02c-90305edad06f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.400176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.406602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.407227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.407511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.407789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.422056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.434295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.434913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.435194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.435479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.436256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] RBAC: Adding query filter params (group.id=9e11188f329f4f03971d4f6a8343c73d, role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=c46131bba4b1483ba24d73c34490f88b) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:08.436563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.438044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.443942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 224/451] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/role_assignments?group.id=9e11188f329f4f03971d4f6a8343c73d&scope.project.id=c46131bba4b1483ba24d73c34490f88b&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:08.477921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.479423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.583467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 228/452] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:58:08.589149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.689509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaacee-d01b-49dc-aa79-236fcc1b3ad9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.723269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.723920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.724203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.724485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] PATH_INFO: `/v3/projects/66b6795243644c5d8ae6b99e815d8914` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.725645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] RBAC: Authorizing `identity:get_project(project_id=66b6795243644c5d8ae6b99e815d8914)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.727156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.729022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 225/453] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/projects/66b6795243644c5d8ae6b99e815d8914 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:08.734489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.782101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.835897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b54e5e-d200-4b1e-b396-d54fe8b23aa3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:08.867955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:08.868615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:08.868896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:08.869173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] PATH_INFO: `/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:08.870403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] RBAC: Authorizing `identity:get_group(group_id=c0c7f1259348472ca5b3ca4095c38b7f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:08.871885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588dbb2-e0c2-4f2d-957f-48c6f34ae882 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:08.873545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 229/454] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/groups/c0c7f1259348472ca5b3ca4095c38b7f => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:08.880369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:08.916615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:08.985688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1da84-35c2-4e81-ba6c-4230cf3e5a3d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:09.017598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:09.018251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:09.018526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:09.018795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:09.019568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] RBAC: Adding query filter params (group.id=c0c7f1259348472ca5b3ca4095c38b7f, role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=66b6795243644c5d8ae6b99e815d8914) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:09.019872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:09.021308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66140-fff0-42b7-b291-c1e01877738c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:09.027233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 226/455] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Jul 28 08:58:08 2022] GET /identity/v3/role_assignments?group.id=c0c7f1259348472ca5b3ca4095c38b7f&scope.project.id=66b6795243644c5d8ae6b99e815d8914&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:11.502816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ddad-e9e4-4c29-b694-8d05c757f2eb None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.503347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ddad-e9e4-4c29-b694-8d05c757f2eb None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.503666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ddad-e9e4-4c29-b694-8d05c757f2eb None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.505490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 230/456] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:11.511293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4532708-c46b-4eb3-ba75-28264ad0f929 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.512167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4532708-c46b-4eb3-ba75-28264ad0f929 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.512639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4532708-c46b-4eb3-ba75-28264ad0f929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.542961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-e4532708-c46b-4eb3-ba75-28264ad0f929 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:11.544211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e4532708-c46b-4eb3-ba75-28264ad0f929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:11.627340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 227/457] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:11.666542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37ddad-e9e4-4c29-b694-8d05c757f2eb None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.808267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37ddad-e9e4-4c29-b694-8d05c757f2eb None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:11.826038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:11.857403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:11.858089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.858373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.858677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.875498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:11.877140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:11.884376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dfa1-ee7b-4d9f-8c6d-7ff77ac2d560 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.884665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dfa1-ee7b-4d9f-8c6d-7ff77ac2d560 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.884944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dfa1-ee7b-4d9f-8c6d-7ff77ac2d560 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.886366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 228/458] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:11.893596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:11.893596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:11.893596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:11.914649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:11.915541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:11.916357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:11.916357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:11.916357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:11.916357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:11.922157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 231/459] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:11.926151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:11.928931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:11.932347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:11.986069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.068476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 229/460] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:12.092690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949c351-e69d-443a-be34-808f5c0cfc4a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.104854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.132703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.133332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.133613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.133907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.135719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:12.136494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.138273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42194f-b125-4360-a146-a7d213ca1cd9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.155315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 232/461] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:12.156884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60feba9-68d7-499a-900d-dc56e441a9e2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.157317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60feba9-68d7-499a-900d-dc56e441a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.157737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60feba9-68d7-499a-900d-dc56e441a9e2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.159377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 233/462] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:12.164206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60feba9-68d7-499a-900d-dc56e441a9e2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.191925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.246060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dbada-0226-4f60-abf7-480b298dbf26 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.282286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.282943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.283217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.283486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.286484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60feba9-68d7-499a-900d-dc56e441a9e2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.293859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.295356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-4f579309-1e89-478e-8d7f-94419f45ed31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:12.314149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:12.315031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:12.315847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:12.315847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:12.315847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.315847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:12.317449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 230/463] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:12.323242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052529a-4843-4a7c-bfbd-502fc773ce0e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.323534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052529a-4843-4a7c-bfbd-502fc773ce0e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.323747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052529a-4843-4a7c-bfbd-502fc773ce0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.325441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.326097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.326441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.326654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.327334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.329381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4c4a0-74fb-4645-9a29-ac30b57b32e2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.349091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d052529a-4843-4a7c-bfbd-502fc773ce0e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:12.350944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d052529a-4843-4a7c-bfbd-502fc773ce0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:12.351889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 234/464] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:12 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:12.353381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f5c-f104-4b3d-9928-dc9c3c24784c None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.353900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f5c-f104-4b3d-9928-dc9c3c24784c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.354568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f5c-f104-4b3d-9928-dc9c3c24784c None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.361524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 235/465] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:12.362794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5f5c-f104-4b3d-9928-dc9c3c24784c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.366198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.473014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 231/466] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:12.473014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.473014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.473014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.493980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:12.495432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:12.552391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.586492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 232/467] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:12.587393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.601046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5f5c-f104-4b3d-9928-dc9c3c24784c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.641672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.642331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.642612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.642885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.643920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:12.644480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.646288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.661226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 236/468] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:12.662475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.717400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.772366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b93c4-81ef-4162-9895-d118fc0db442 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.795283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b614-4e1d-407d-b2cb-f5ac795328a1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.810005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.811082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.811582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.812051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.823190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.825003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.835819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:12.837211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:12.837892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:12.838366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:12.841635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:12.842478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:12.843874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:12.843874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:12.843874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.843874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:12.848549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 233/469] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:12.849501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.854480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:12.856210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:12.866191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:12.874010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:12.875232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:12.876264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:12.876264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:12.876264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:12.876264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:12.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 237/470] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:12.879757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:12.966200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49d9e4-f893-488c-87f8-5e45bb4f7d88 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:12.989383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:12.996287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5702-d729-4e2d-9530-f4bbd2ff22a0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.002053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.002688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.002968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.003240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.003914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.005805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.033873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.034512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.034795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.035125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.035770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:13.036074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.037642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9049d9d-19d9-42d7-b146-0c892b5892d7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.038372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 234/471] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:12 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:13.039368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.052881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 238/472] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:13.054865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6847-f083-4d7a-8b8b-058ff283b7d5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.055398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6847-f083-4d7a-8b8b-058ff283b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.056137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6847-f083-4d7a-8b8b-058ff283b7d5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.058187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 239/473] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:13.059241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6847-f083-4d7a-8b8b-058ff283b7d5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.130796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:13.190928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11874bd-e45b-456d-8453-24bfea23caba None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.204116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6847-f083-4d7a-8b8b-058ff283b7d5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.233997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.234589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.234865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.235303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.235856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:13.236235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.237531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030dbf-f7b7-4db0-be12-f16ff9703c68 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.247948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.248575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.248863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.249174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.250379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.251755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 235/474] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:13.252909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.253033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.253128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.253484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.273910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:13.275222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:13.280032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 240/475] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:13 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:13.281416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.329508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:13.388212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 236/476] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:13.413168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.431428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45fe9a-ae7f-4de8-95c8-120fdfb96b36 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.442203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:13.473074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.473829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.474126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.474398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.475137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.476777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.516961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 241/477] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:13 2022] POST /identity/v3/users => generated 248 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:13.571024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30b33-8649-4790-a81a-81622987ce4e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.631873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.632541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.632804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.633085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.633959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:13.635504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:13.650266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.sql.core [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:58:13.650266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:58:13.650266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [parameters: {'id': '8cf94a3f24354c968b60aa28990d9962', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:58:13.650266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80374) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result.read() Jul 28 08:58:13.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:58:13.670849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application yield resource Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise self.value Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application session.commit() Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:58:13.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:58:13.673865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:58:13.674740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result.read() Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:58:13.675531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [parameters: {'id': '8cf94a3f24354c968b60aa28990d9962', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:13.676295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 08:58:13.678586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:13.685667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 237/478] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 28 08:58:13 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:13.685807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.811944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7215fe-bbba-4f2e-aef0-17ae6ea06c23 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:13.855663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:13.856973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:13.857456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:13.857979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:13.871849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-44d0ee2c-e926-41a0-af54-a124883eb9fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:13.877566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:13.878683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 08:58:13.879766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:13.882515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 242/479] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:13.891411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:13.937745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:14.029636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7087af-afe5-4339-93b3-b7fc459c4a43 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:14.066735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:14.067376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.067655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.067932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.068661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:14.068968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:14.070397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b410e86-7e3c-48a0-8522-0cb237b70bb8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:14.079709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 238/480] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:14.856305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5e2a-9a50-441a-93d1-9cc5d17422e2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.856836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5e2a-9a50-441a-93d1-9cc5d17422e2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.856836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5e2a-9a50-441a-93d1-9cc5d17422e2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.858616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 243/481] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:14.864775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afffa-d276-481c-8662-ed3812237880 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:14.865258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afffa-d276-481c-8662-ed3812237880 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:14.865331 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afffa-d276-481c-8662-ed3812237880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:14.891406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-888afffa-d276-481c-8662-ed3812237880 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:14.892627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-888afffa-d276-481c-8662-ed3812237880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:14.995512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 239/482] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:15.026463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b5e2a-9a50-441a-93d1-9cc5d17422e2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.064117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:15.169501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b5e2a-9a50-441a-93d1-9cc5d17422e2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.208955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.210626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.211418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.212189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.229580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:15.236224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:15.237710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:15.238829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:15.241628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 244/483] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:15.247098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888afffa-d276-481c-8662-ed3812237880 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.277396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:15.384774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888afffa-d276-481c-8662-ed3812237880 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.436309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.437403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.437687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.437988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.438718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:15.439020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:15.440404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289856a-fd4e-4819-b314-d553ae808809 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:15.449853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 240/484] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:15.458858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.587506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:15.606227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c86f0-5e02-49e3-9e95-ef4e5dae90f2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.654461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.655106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.655386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.655696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.669868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d3e1a-3610-4751-9a41-1f17fe9dc96e None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.670494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d3e1a-3610-4751-9a41-1f17fe9dc96e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.670968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d3e1a-3610-4751-9a41-1f17fe9dc96e None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.672074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:15.673545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 241/485] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:15.673679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:15.679252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.679822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.680324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:15.703742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:15.704760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:15.705603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:15.705603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:15.705603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:15.705603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:15.707545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 245/486] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:15.712189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:15.713467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.715880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:15.861298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 242/487] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:15.884748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:15.909469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa9dae-e197-46ba-9803-b1343cf25c3b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:15.913886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:15.993611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:15.994749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:15.995232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:15.995708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:15.996628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:15.997147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:15.998753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:16.003408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.029175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 246/488] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:16.035272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.088076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c00b52-0131-4944-bbda-ec9385d06db6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.132635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.135102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.135591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.137238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.162561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:16.173028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:16.173984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:16.174777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:16.183849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 243/489] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:16.192306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.229088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518b5d4-e65b-4b72-b8e7-4c95952d7c87 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.271443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.271855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.272105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.272353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.281366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:16.282969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-4c153527-2f9e-4d32-a127-c6e4f323fd45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:16.301325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:16.302550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:16.303416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 247/490] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:16.304605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618e66-f6e5-40b5-b9c5-e737b7fc07f8 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.304936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618e66-f6e5-40b5-b9c5-e737b7fc07f8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.305275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29618e66-f6e5-40b5-b9c5-e737b7fc07f8 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.311743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 248/491] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:16.312697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29618e66-f6e5-40b5-b9c5-e737b7fc07f8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.331132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.362307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275018b-a7b3-4e07-9481-31aca5246c0a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.391261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.418901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.419748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.420132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.420520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.421393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:16.421793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:16.424105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e604ad-b5a9-4a48-b460-d8c395623aae None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:16.440985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 244/492] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:16.442436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.442724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.443002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.462026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:16.462218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29618e66-f6e5-40b5-b9c5-e737b7fc07f8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.463256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:16.500663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.501302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.501586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.501892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.502704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:16.503011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:16.504475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa662b9-6adc-47c6-9442-57324ad26483 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:16.539192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 245/493] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:16.540046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.564167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 249/494] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:16.565770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35288e-bcd2-4872-b5b4-4d324f983ef9 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.566112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35288e-bcd2-4872-b5b4-4d324f983ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.566391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35288e-bcd2-4872-b5b4-4d324f983ef9 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.569295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 250/495] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:16.569295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35288e-bcd2-4872-b5b4-4d324f983ef9 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.640343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.681749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ebdc-ac50-40cc-bf4c-797edc16436c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.712092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35288e-bcd2-4872-b5b4-4d324f983ef9 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.721917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.722900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.723412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.723864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.727816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.736119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:16.737878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:16.749002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.750227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.750507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.750774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.761662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:16.762281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:16.763075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:16.763893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:16.763893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:16.763893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:16.763893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:16.770076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 246/496] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:16.770076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-791414b4-79de-409a-b7d1-4b675ea290ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:16.770323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:16.771101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:16.771908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 251/497] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:16.774732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.775067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.775351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.794143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:16.795374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:16.866383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 252/498] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:16.867244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:16.909884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0570-128f-43ac-81bb-5767f63456ae None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:16.934447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:16.955750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:16.956385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:16.956662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:16.956937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:16.998648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e14737-3d30-4554-b808-31c9ad13a2ab None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.001950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.034620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.035251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.035541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.035818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.036521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.037310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.038317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.039576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.041040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.050523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 253/499] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.051118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.149137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 247/500] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:17.150039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.169303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe05e42-e1f6-4e8a-b783-4b984967be42 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.172820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.216898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.217571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.217804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.218105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.218820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.219123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.220500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d9445-62f8-4bf0-b8bb-e52ee50eb5b5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.228718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 254/501] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.230066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c7d6-fe36-49de-96ee-78996df12760 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.230374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c7d6-fe36-49de-96ee-78996df12760 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.230623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c7d6-fe36-49de-96ee-78996df12760 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.231870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 255/502] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:17.232571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c7d6-fe36-49de-96ee-78996df12760 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.262604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a43779-02b0-4a9b-b614-ca2332797a8f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.269090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.314347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.314899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.315174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.315442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.326033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:17.331130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:17.332036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:17.332816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 248/503] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:17.333458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.353973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c7d6-fe36-49de-96ee-78996df12760 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.394014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.394656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.394937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.395209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.403191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.404656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:17.416142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:17.417115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:17.418012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 256/504] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:17.418557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.427502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.452291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c00e5-7b93-43b4-ba68-a7a5de499955 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.488376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.489015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.489292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.489559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.501065 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.502571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.507743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-39a456f6-b576-4ec0-ae38-428c67443b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:17.518294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:17.519625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:17.520674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:17.520674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:17.520674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:17.520674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:17.520674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 249/505] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:17.520997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.521276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.521658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.527975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334bf22-bcfc-4c97-9cb8-89b09be7c328 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.540878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:17.542133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:17.563116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.563762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.564040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.564314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.580190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.581592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:17.597221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:17.598304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:17.599230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:17.599230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:17.599230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:17.599230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:17.603059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 257/506] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:17.603921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.611879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 250/507] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:17.612742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.698706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.727251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88d6-dbf4-4066-9f5e-6588c8801cac None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.745256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f0655-583f-42d7-83ac-cb90f56e3068 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.763109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.763727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.764013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.764289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.765633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.765968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.767353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.772703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.777003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 258/508] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.778213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.785776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.786441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.786722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.787001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.787817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.788125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.789572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.827441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 251/509] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.828647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.905012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae94734-a2a3-4289-ad0f-d527504b1cb0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.943256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e74a6b-cc9e-49e2-b75d-220132155bb3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:17.946042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.946691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.946968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.947240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.947945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.948251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.949657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:17.961792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 259/510] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:17.962921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:17.975986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:17.992612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:17.993242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:17.993521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:17.993796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:17.994522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:17.994828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:17.996208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.009130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 252/511] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:18.010171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.032177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.077416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27d53e-31bb-48c1-9fc4-c9caa75b4208 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.111420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.112069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.112352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.112623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.115527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab25a5-099c-4a50-b9d4-c7c102dfbdfd None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.124965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.126465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:18.141721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:18.142686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:18.143512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:18.143512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:18.143512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.143512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:18.143512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 260/512] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:18.143772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.155557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.156134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.156416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.156687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.167129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.168618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:18.186225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:18.187938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:18.188955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:18.188955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:18.188955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.188955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:18.188955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 253/513] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:18.189270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.223650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.258365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3149992-e98f-4cdd-9f13-338c9ae857e4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.271583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.293970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.294599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.294881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.295153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.308704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6eabd-71c5-439e-bd11-aa68bec85a27 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.344994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.345639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.345944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.346221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.353262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.354872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.366743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.368230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:18.368579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:18.369471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:18.370341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 254/514] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:18.370909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.464246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.473579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549cae3-0878-4e5f-8cc7-ee68b47413e6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.474599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 261/515] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:18.474599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.518193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.519135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.519424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.519854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.532776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.534302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:18.548976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:18.550371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:18.551233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 255/516] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:18.551787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.589319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c0f35-de05-493f-a957-ba713201080a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.626577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.627195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.627469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.627738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.628429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:18.628725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.630148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.641963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 262/517] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:18.642799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.675384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19806760-589c-4c25-83ec-f1805ec2dc41 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.712821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.713455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.713743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.714052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.714805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:18.715238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.716641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.729427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 256/518] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:18.730304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.737900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.750151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:18.772593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107724-1c0b-4f6a-a717-70ea56b1a0d6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.806959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.807581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.807862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.808133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.808883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:18.809187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.810664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.829831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 263/519] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:18.830820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.852129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957807c5-bc93-412e-87b0-59f2091d4db3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.895608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:18.896259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:18.896549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:18.896821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:18.908231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:18.909712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:18.928858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:18.930953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 257/520] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:18.932447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:18.963159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b906c1-3d83-43c1-8a00-e89f798c63db None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:18.994499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.000998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.001149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.001676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.001935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.002256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.003259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:19.003571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.005047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.024975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 264/521] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 28 08:58:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:19.025971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.053715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe2b35-3bd0-4f78-81f6-437ec23c21a3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.095119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.095739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.096019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.096292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.096976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.098483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.119485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 258/522] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:18 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:19.120319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.128040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01a28-312a-4460-958d-6e13477c4ffe None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.173402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.174269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.174636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.174965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.183979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.185640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:19.209041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:19.210245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 265/523] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:19.212764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.277604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e62b-5c8f-4925-87cc-fb399dd6416c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.287649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.306857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.327245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.328159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.329187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.329187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.330512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:19.330512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.332988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.340830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 259/524] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:19.341780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.370102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636696-9f2f-449b-b47b-1b1b0c0ae435 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.425931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.426992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.427485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.427986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.429342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:19.429948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.431941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.449718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 266/525] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:19.451009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.486046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769d8fc-4112-4ee6-be02-505e3bb9851a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.526438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.527074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.527358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.527683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.583618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820114-1e39-4f59-837c-fdff0fce8005 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.591781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.610599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.611447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.612751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.645461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.646447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.646921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.647953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.649268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:19.649753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.651478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.677605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 267/526] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:19.678783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.766002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 260/527] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:19.767159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.804437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8597206-a3f2-46fd-bee9-53b82a860f07 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.843600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.844752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.845208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.845664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:19.854143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:19.855905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:19.873690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:19.878687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:19.879570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:19.880483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 268/528] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:19.885555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:19.903163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ff35-b167-489c-b0e9-021a44b5ab09 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:19.928087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:19.958274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:19.959345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:19.959824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:19.960295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.038747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ac002-ff42-4934-8c5b-5d41ca5e2d11 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.044734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.046839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.081274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.082464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.082949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.083469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.096773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.099146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:20.121314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:20.122725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:20.123577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:20.123577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:20.123577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:20.123577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:20.126343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 269/529] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:20.127494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.160160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.179082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 261/530] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:20.180383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.227840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.251705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39dfaa-9d83-476f-b5b4-736e4edb7693 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.289108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8c72-5b1f-4199-8baa-f658338c2551 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.291269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.292163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.292659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.293148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.294136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:20.294662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.296333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.315719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 270/531] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:20.316959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.328195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.328881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.329166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.329496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.330345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:20.330707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.332356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.351967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 262/532] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:20.353080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.418305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.434928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f23d6-f19a-4f12-8837-b12abf2bcd2e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.474078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.475208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.475762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.476233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.483021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85831b-a434-43b1-b630-3654ac87aca7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.487281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.491437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.494067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:20.513421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:20.514400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:20.516062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:20.516062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:20.516062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:20.516062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:20.518888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 271/533] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:20.521321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.524131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.525062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.525561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.525872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.526731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=40d10d1a51524ba2967695566bb78e42) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:20.527867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.528528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.534840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 263/534] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/role_assignments?user.id=40d10d1a51524ba2967695566bb78e42&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:20.535814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.644033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc746f-6497-4058-99f1-7397dedd7aa2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.666252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7985ff-6554-49bb-87ee-6b3aa43e6faf None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.677602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.711525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.712504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.713001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.713481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.714514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:20.715053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.716754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.731797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 264/535] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:20.737212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.763680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.764674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.765707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.766214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.781699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.783225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:20.804478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:20.805375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:20.806501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 272/536] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:20.813525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:20.878466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:20.934294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1905898-c285-42e6-810f-eff43979ebd1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.945979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6b09-91e6-4d03-99da-101f12fb2a39 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:20.982661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:20.984260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:20.984613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:20.984960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:20.986033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:20.986415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:20.988810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:21.007690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 273/537] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:21.010320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:21.038591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:21.039378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.039675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.039961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.053354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:21.059523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:21.082845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:21.083746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:21.084569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 265/538] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:21.095360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:21.123640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01719-c480-4b91-ad5a-517f984eee91 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:21.144187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:21.151124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:21.190527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:21.191497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.191949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.192432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.193497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=6077e3f936ea42acbc6879758ddc8336) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:21.194040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:21.195796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:21.210164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 274/539] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:21 2022] GET /identity/v3/role_assignments?user.id=6077e3f936ea42acbc6879758ddc8336&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:21.246612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc9325-3ede-45d4-bbc6-8b6b40a28654 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:21.301305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:21.301975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.302264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.302544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.303321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:21.303632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:21.305148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b48f-bc2b-447e-8399-05d35f10a656 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:21.325427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 266/540] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:21.331127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:21.468810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb98eac-ef7e-4273-b393-7739de9cdb4a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:21.504655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:21.505654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.506160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.506626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.507614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e4e8151c2541400cb76b8b19384f49d8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:21.508141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:21.509799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1c526-c804-458b-b053-6e7755e55fb5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:21.516544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 275/541] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:21 2022] GET /identity/v3/role_assignments?user.id=e4e8151c2541400cb76b8b19384f49d8&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:21.802172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc47e-1cb4-4061-8b0c-9ad51b0be1f3 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.802546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc47e-1cb4-4061-8b0c-9ad51b0be1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.802739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bc47e-1cb4-4061-8b0c-9ad51b0be1f3 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.804382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 267/542] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:21.811989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160087ed-3051-4383-9af8-9f508acf5484 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.812572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160087ed-3051-4383-9af8-9f508acf5484 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.813042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160087ed-3051-4383-9af8-9f508acf5484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.837571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-160087ed-3051-4383-9af8-9f508acf5484 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:21.839184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-160087ed-3051-4383-9af8-9f508acf5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:21.878632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:21.914531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 276/543] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:21.937270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aac16-3fcd-4454-8b4d-58f99159b33b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.937900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aac16-3fcd-4454-8b4d-58f99159b33b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.938378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aac16-3fcd-4454-8b4d-58f99159b33b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.939929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 268/544] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:21.947089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:21.947383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:21.947662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:21.953547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aac16-3fcd-4454-8b4d-58f99159b33b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:21.972462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:21.973679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:22.057506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 277/545] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:22.082123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.150656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:22.175902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aac16-3fcd-4454-8b4d-58f99159b33b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.198692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:22.215879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f4108-7037-4632-b60e-f8d11a0a8dfe None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.232357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.233028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.233309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.233632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.261067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.261727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.262035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.262307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.263836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:22.269919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:22.270825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:22.271593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:22.277475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 269/546] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:22.278407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.284410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:22.289593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:22.291721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:22.293747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 278/547] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:22.297937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.441392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7046-aa91-4e12-ba61-dc2839c7bac3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.477166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.477794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.478115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.478399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.479124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:22.479429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:22.480817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:22.497789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 270/548] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:22.506475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.508736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601cdaf-8df9-4301-95e5-64cc14e53087 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.530141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:22.610357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.611002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.611357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.611692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.612544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:22.612980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:22.614442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:22.619686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e066-37af-4f3f-afea-956e44bced98 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.637324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 279/549] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:22.643587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.662859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.663749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.664031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.664306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.689131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:22.690647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:22.694141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:22.728618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:22.731260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:22.733610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:22.733610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:22.733610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:22.733610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:22.744905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 271/550] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:22.745881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:22.873709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b677f8-5cbc-4e8f-811a-76f94523b527 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.874734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:22.956788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16104c2-16a5-4c4f-99c6-4e7c5e51ba6d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:22.963966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:22.964643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:22.964927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:22.965203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:22.989917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:22.991413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:23.033782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:23.034722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:23.035733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:23.035733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:23.035733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:23.035733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:23.038014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 280/551] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:23.042845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.043775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.044263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.045359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.046718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.049685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:23.050245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.051872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.086350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 272/552] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:23.090682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.156146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.192346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bddf0-6c60-4a61-815b-0de31faa2772 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.219651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ca73-a366-4eab-af5d-32c5fa21ea45 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.238428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.239235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.239613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.240003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.241205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:23.241867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.243668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f35402-2dcb-4b60-a8b5-1cd560aaa622 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.247867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.258437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 281/553] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:23.260010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac285-9b27-411a-97e5-4420dd5913ce None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.260528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac285-9b27-411a-97e5-4420dd5913ce None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.260929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac285-9b27-411a-97e5-4420dd5913ce None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.266668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 282/554] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:23.266668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bac285-9b27-411a-97e5-4420dd5913ce None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.268365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.269234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.269703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.270226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.279733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.281782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a075ef6e-58cb-49d2-8eed-0cd5e8ffcd1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:23.295745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:23.297359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:23.298342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:23.301406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 273/555] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:23.303267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.303929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.304410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.326969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:23.328607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:23.386557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bac285-9b27-411a-97e5-4420dd5913ce None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.412419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 274/556] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:23.413710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.426838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.426838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.426838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.426838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.427672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.437627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.439291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:23.458488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:23.459547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 283/557] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:23.461524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.558777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0f658-20a7-4617-91f4-30d6405a8e9c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.573629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db1ab-02a9-4cd2-9784-83c6bc76a197 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.574665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.599325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.600431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.600943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.601421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.602638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:23.603252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.604971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.611823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.612485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.612770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.613050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.624725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:23.627537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 275/558] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:23.628734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:23.633460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:23.634531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:23.635733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 284/559] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:23.636424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.726106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.749947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e42e-05a4-495c-833b-cf68d91304ea None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.775025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68227a-d728-40dd-bd74-af1a0ebc2d1d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.788836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.789932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.790510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.790983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.792115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:23.792631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.794457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.813323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.813907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.814200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.814615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.816126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 276/560] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:23.817398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:23.869204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.911784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.913252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c24262-440a-4e1a-80b9-68ee5858306c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.930089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be158db9-40f8-4654-bf95-2314b1c8f1e0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:23.967098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:23.967736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:23.968576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:23.968859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:23.969589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:23.969922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:23.971307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:23.972746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:23.980385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 277/561] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:23.981073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.024678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 285/562] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:24.026042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bf75-efb4-4a25-bc30-09a3c640951e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.026332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bf75-efb4-4a25-bc30-09a3c640951e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.026616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bf75-efb4-4a25-bc30-09a3c640951e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.028036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 286/563] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:24.032043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd037de-149c-46aa-a053-9401897fdb8b None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.032626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd037de-149c-46aa-a053-9401897fdb8b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.033108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd037de-149c-46aa-a053-9401897fdb8b None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.034936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 287/564] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:24.035674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd037de-149c-46aa-a053-9401897fdb8b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.092650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.106055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3810aa4-4193-412e-b515-79e24236abe1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.140332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.140980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.141257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.141536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.142942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd037de-149c-46aa-a053-9401897fdb8b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.152658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.154160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:24.169223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.170883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 278/565] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.172340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.186277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.187298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.187773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.188243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.205292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.208099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d056a8b4-8f5c-4e1a-aa2c-0f897b06fbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:24.235683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.236658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.237508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:24.237508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:24.237508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.237508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:24.242146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 288/566] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.243553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.243844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.244121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.257558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.264244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:24.267164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:24.276092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51e1d-52e3-4b74-93a4-75cdc35c055a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.310051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.310712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.310995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.311272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.322786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.324249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.326112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-be1dd4e8-2fa8-4c75-9490-55d75a7f04c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:24.341979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.343222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.344103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:24.344103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:24.344103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:24.344103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:24.344103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 279/567] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.344508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.344757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.345042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.349058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 289/568] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:24.350687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.366810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:24.368022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:24.437270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 280/569] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:24.438162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.464096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f7cca-e2fa-4cac-888e-761a5f4ca162 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.500621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.501255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.501535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.501808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.502538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:24.502843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.504229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.506694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.517291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 290/570] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:24.518260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.553246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e14374-22f0-4ec0-90a6-5b8cabd2f71a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.577165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.592462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.593123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.593536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.593666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.594403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:24.594707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.596090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.615063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 281/571] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:24.615063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.620111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc6683-89ec-4673-a4d5-6e3998fd69dd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.656197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.656879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.657224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.657509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.658251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:24.658565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.659947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.676238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 291/572] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:24.676238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.725045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb65c7-ebdd-4277-8bf8-e185d6568f71 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.733071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.761159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.761808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.762115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.762383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.776906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:24.781266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577432ce-e42c-49f3-b81f-6572e80fd8f0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:24.786558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.787989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:24.789072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 282/573] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.789739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.820618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.821258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.821539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.821812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.832630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:24.837621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.838784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:24.839633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 292/574] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.840344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.844986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.910724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac9c6c-46d8-47ae-8906-5bbe7b058f6b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.953435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.954113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.954387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:24.954656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:24.957533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586afbc-2356-4737-838c-c8e18cabc735 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:24.961560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:24.963052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:24.974961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:24.976046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:24.977025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:24.977717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:24.978006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 283/575] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:24.978902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:24.998324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:24.998978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:24.999806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.000084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.007814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.009279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:25.021172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:25.023229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 293/576] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:25.026547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.095787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1741f-e603-45a7-bc8f-7757582d1ad0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.098096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.137635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.138319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.138599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.138876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.140547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613898-5424-4449-8efe-caefc6307a9c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.146052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.147514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:25.161161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:25.162244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 284/577] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:25.163484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.181188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.181817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.182482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.182482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.183150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:25.183458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.184848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.193097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 294/578] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:25.193981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.234963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.274814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d3e0e-2762-4124-8f29-bcdbc91ae34a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.302787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358ce6b-968b-4a4e-80ee-66c1297b17a0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.311182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.311844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.312129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.312397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.313142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:25.313442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.314850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.328071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 285/579] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:25.328071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.337473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.338131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.338412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.338686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.339464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:25.339750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.341190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.354147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.354316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 295/580] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:25.354709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.437004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3efd-7b14-4546-a899-a32509325901 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.455722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.460097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c33384-2394-494f-b748-fedc483bd766 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.480398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.481041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.481324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.481603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.482452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:25.482764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.484280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.502999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 286/581] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:25.503885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.506363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.507107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.507595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.507970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.508745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:25.509058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.510534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.523204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 296/582] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:25.524059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.605052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.621230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19c1a-fd37-4339-8be5-e54d97f28965 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.641961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793bf6-66f1-475f-93a6-fcbfbc5b9355 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.656083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.656737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.657007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.657286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.668092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.669586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.677002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.677929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.678316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.678759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:25.687711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:25.688786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:25.689701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:25.689701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:25.689701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.689701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:25.689701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 287/583] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:25.690009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.697562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.700059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:25.715659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:25.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:25.717669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:25.717669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:25.717669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.717669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:25.717669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 297/584] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:25.718621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.731431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.798729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5262bc-7fb7-4c60-9ad7-956f3a696795 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.836172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e77b3-a326-4138-bd30-40751893bdd7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:25.843898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.844578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.844855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.845128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.848661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:25.860853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=96a7013052fd4d48a2463cf51f13861f, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.862290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.873871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:25.875091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:25.875367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:25.875643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:25.878494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 288/585] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:25 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:25.879411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.892244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:25.893781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:25.913227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:25.914283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 298/586] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:25.915151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:25.996444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.026889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251b29-e44d-4a4e-b171-490a67c950f5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.056452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a191-fb6d-4aa9-8a9e-9b4c22034829 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.082061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.083008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.083298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.083576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.084388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=59f92dd2b31f4a6dbb732ee67575ac1f) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.084738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.086195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.092870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 289/587] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/role_assignments?user.id=59f92dd2b31f4a6dbb732ee67575ac1f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.093707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.099692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.100338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.100622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.100898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.101661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.102005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.103438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.123618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 299/588] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.124980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.171673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.243138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cc8da-0b60-452e-9790-07ccf7693db1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.255641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe153ee-4f23-4d6d-ab1c-a84dc1595c88 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.282281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.282538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.282958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.283241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.283517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.284225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.284532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.285958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.294909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.295789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.296780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.297271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.298386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.298842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.300384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.300873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 290/589] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.301751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.315641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 300/590] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.316931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.412765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539853b-b28f-428b-a605-c5315f620ac0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.433868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.434432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138f7e-4869-46b0-9cba-7e6a9ce9d13c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.451163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.451824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.452102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.452410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.460432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.461926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.474352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.475141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:26.475555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:26.476480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:26.477404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:26.478689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.478689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.478848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 291/591] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:26.478848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.490194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.491831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:26.509764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:26.512001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 301/592] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:26.514247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.578038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.592290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeada2-e6bd-41b3-a719-853bc3fbc3f5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.629504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.630217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.630524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.630922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.638288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.639824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.651121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fbc7f-a3ac-4634-8230-cead14bdd46c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:26.651913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:26.652776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:26.653632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 292/593] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:26.654407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.699297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.700793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.701291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.701749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.703202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.704046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.706097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.726318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.735387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 302/594] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.735387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.795146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ffd66-ba46-4ed1-ad86-9f85f386bea4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.841834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.842607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.843416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.843696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.844515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.844823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.846813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.873932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 293/595] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.875386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.880854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:26.885857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2533a15-aa6b-4a26-9129-725d90e0fc94 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:26.933328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:26.934347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:26.934824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:26.935315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:26.936302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:26.936916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:26.938657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:26.953577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 303/596] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:26.954757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:26.999832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.015623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce954014-6341-4038-ae63-c2af3e49baf4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.052362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.053014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.053296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.053570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.065047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.066582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.079015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46e3c8-a2fd-45b7-bd10-62527e0d30a3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:27.082598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:27.083491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:27.084492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:27.084492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:27.084492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:27.084492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:27.088577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 294/597] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:27.089581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.123419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.124630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.125107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.125622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.138063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.140696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:27.158212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:27.159151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:27.160057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:27.160057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:27.160057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:27.160057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:27.166893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 304/598] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:27.168253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.185861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.197493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da603-b1f9-4829-82d2-21023255f269 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.233857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.234469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.234845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.235029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/40d10d1a51524ba2967695566bb78e42/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.246587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.251491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=40d10d1a51524ba2967695566bb78e42, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.252711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.270431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 295/599] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:27 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/40d10d1a51524ba2967695566bb78e42/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:27.271460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.293530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c78d5-e805-49ae-926b-37bf9dcf1d22 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.338308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.339471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.340058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.340575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.341595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:27.342188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.344160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.359344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 305/600] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:27.376251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.393895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9693e9-d497-4011-8901-8651b8bac61f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.433198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.433990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.434212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.434487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.435275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:27.435584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.441885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.451382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 296/601] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:27.458375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.519901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.525030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.562837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f59d7-c9dc-450e-a3a2-e0056b1d37fb None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.609661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe2a9-0035-483b-87a0-4b584cda9dab None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.629352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.630428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.630906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.631372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.638823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.641137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:27.653552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:27.654935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:27.655833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:27.658353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 306/602] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:27.666873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.695907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.696567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.696851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.697124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.710721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.712197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:27.738652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:27.739591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:27.740582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 297/603] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:27.747958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:27.855557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ed2d6-525c-41f7-bad5-13217abfd716 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.897636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8b3d-4b68-4d9e-a559-4e235b96c794 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:27.914972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.954865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:27.962766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.963801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.964267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.964727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.966058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:27.966641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.969209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.975171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:27.975829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:27.976114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:27.976390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:27.977162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:27.977475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:27.978954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:27.992813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 298/604] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:28.001044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:28.017708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 307/605] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:28.018722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:28.139331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082d4c-0dff-4ddd-93bb-9b9f1fce9089 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:28.178266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:28.178963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.179245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.179524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e4e8151c2541400cb76b8b19384f49d8/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.193359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0bea-6c2d-4ca4-8f17-566c83937d11 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:28.195830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=e4e8151c2541400cb76b8b19384f49d8, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:28.197300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f1de-062e-41d8-b16d-45a5be478c6f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:28.217221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 308/606] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:28 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e4e8151c2541400cb76b8b19384f49d8/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:28.219471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4f0-4c64-4814-a8d2-bbcdef226334 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.219761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4f0-4c64-4814-a8d2-bbcdef226334 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.220069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4f0-4c64-4814-a8d2-bbcdef226334 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.221934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 309/607] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:28.228430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.228707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.228979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.238229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:28.239124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.239530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.239915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/6077e3f936ea42acbc6879758ddc8336/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.248957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:28.250295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:28.263681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=6077e3f936ea42acbc6879758ddc8336, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:28.266898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:28.303468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 299/608] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:27 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/6077e3f936ea42acbc6879758ddc8336/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:58:28.321279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:28.364865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 310/609] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:28.415958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:28.448324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:28.559823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a131a7-f153-46ae-b313-ce877bf40f22 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:28.605672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:28.606420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.606711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.606998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.620124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:28.625507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:28.626581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:28.627426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:28.630145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 300/610] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:28.636143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:28.777873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74ef9-da2d-4cfd-b00f-785cec9c33b6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:28.814365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:28.814999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.815277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.815550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.816277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:28.816579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:28.818036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dae56e-e572-4261-ac63-c9daf1a3d659 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:28.828108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 311/611] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:28.833876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:28.855784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e843de-9f96-4f79-a5b9-f07820556f04 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.856114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e843de-9f96-4f79-a5b9-f07820556f04 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.856337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e843de-9f96-4f79-a5b9-f07820556f04 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.857954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 312/612] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:28.863847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:28.864130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:28.864407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:28.886770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:28.888236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:28.933169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:28.984056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 313/613] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:29.009092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.025272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:29.057982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c65c1-6a3a-46c1-982e-d3adc943667f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.125263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163194-9e6d-4700-b0ea-669c2d96b4e0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.153282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.154056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.154342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.154622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.164924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.165582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.165874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.166166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.177284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.178802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.179751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:29.185383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:29.186405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:29.187807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:29.188430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 314/614] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:29.194269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:29.207668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:29.209810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:29.212020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:29.212020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:29.212020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:29.212020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:29.215290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 301/615] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:29.221379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.221712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:29.372833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdb6b7-86ee-4dd7-9673-1a47d1af7860 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.421457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f051e-c8b6-4fb9-97da-e9c234563e04 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.447860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.448505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.448779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.449054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.451450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:29.451757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.453708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.474284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 315/616] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:29.476650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:29.480247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.486285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.487908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.488194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.488472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.489190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:29.489493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.490909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.516459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 302/617] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:29.527393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.634378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:29.642464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93242-5088-403b-af4a-4086d2b2068b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.728165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.728829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.729112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.729387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.734692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651853c-d8c2-4a10-8396-8f29b44f06a3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.755195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.760766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:29.793908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:29.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:29.796779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:29.796779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:29.796779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:29.796779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:29.799114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 316/618] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:29.801089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.823212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.823880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.824163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.824437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.834505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.836501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-feb43015-f96f-46c6-9f6d-2316665a9579 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:29.856365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:29.857412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:29.859440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:29.868672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 303/619] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:29.871522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d213-7fbc-4b8e-a0e2-97ae65b7e5b9 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.872376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d213-7fbc-4b8e-a0e2-97ae65b7e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.873176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d213-7fbc-4b8e-a0e2-97ae65b7e5b9 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.876269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 304/620] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:29.881540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d213-7fbc-4b8e-a0e2-97ae65b7e5b9 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:29.939981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8910-4aa9-49a9-bf27-c3f68442f7fe None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:29.963814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:29.987101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:29.987804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:29.988086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:29.988363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:29.989120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:29.989430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:29.990858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d61b4-98ea-4b4b-b7e8-dc0a482bc8d3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:29.998844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.005012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 317/621] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:30.006467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.007200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.007588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.027095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d213-7fbc-4b8e-a0e2-97ae65b7e5b9 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.028181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:30.029427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:30.076686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.078727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.079732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.080268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.082085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:30.083169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.085260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.123685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 305/622] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:30.124985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.141876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 318/623] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:30.141876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.261141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431db1b-e7c1-4770-9710-30375e743298 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.261753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d776c3-f080-43d3-967f-854472a1ff51 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.273531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.291860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.302521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.303183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.303464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.303700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.303824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.304679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.305176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.305644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.316141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.318611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:30.335043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:30.336038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:30.336981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:30.345132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 306/624] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:30.345132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.406563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.409277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.482026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac7d84-3414-4574-bf59-19c10302f8a6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.528703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824719-382b-46f8-997a-e7045202994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.529409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824719-382b-46f8-997a-e7045202994b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.529741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824719-382b-46f8-997a-e7045202994b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.530041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824719-382b-46f8-997a-e7045202994b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.549078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 319/625] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:30.549766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00824719-382b-46f8-997a-e7045202994b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:30.550564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-00824719-382b-46f8-997a-e7045202994b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:30.556206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:30.557190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:30.558049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:30.561967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 307/626] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:30.562916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824719-382b-46f8-997a-e7045202994b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.590558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.603748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.675394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11f359-b6b2-48d7-bb3b-942a40240cd6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.715053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.715696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.715973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.716245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.717064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:30.717370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.719499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.744395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 320/627] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:30.744395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.823812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824719-382b-46f8-997a-e7045202994b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.841712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:30.863098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29453-52e2-4ed0-b1b7-449d2cfb3425 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:30.863975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.864703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.865052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.865415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.876628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.878127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-00212d27-1603-438a-81b2-b4c722b1a12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:30.895356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:30.896582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:30.897628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:30.897628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:30.897628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:30.897628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:30.897628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 308/628] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:30.898036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d235b25-8316-479a-b462-7075f7ea7037 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.898036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d235b25-8316-479a-b462-7075f7ea7037 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.898321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d235b25-8316-479a-b462-7075f7ea7037 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.903561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 309/629] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:30.904362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d235b25-8316-479a-b462-7075f7ea7037 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:30.910172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:30.910797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.911078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.911351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.912075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:30.912381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:30.913763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f994-709f-4831-ac1d-1fd1dc1c9fd0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:30.925586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 321/630] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:30.927359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bb08-6146-4690-931b-b7e50388817d None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:30.927838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bb08-6146-4690-931b-b7e50388817d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:30.928194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bb08-6146-4690-931b-b7e50388817d None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:30.930560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 322/631] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:30.932130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374bb08-6146-4690-931b-b7e50388817d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.001324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.025269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d235b25-8316-479a-b462-7075f7ea7037 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.054635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374bb08-6146-4690-931b-b7e50388817d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.064978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.065631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.065940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.066216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.077964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.079335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.090139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.090762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.091041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.091312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.092004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:31.092305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.093923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-21cea3fe-bf1e-48ad-8c81-17f1f33b62d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:31.094523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.095533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.096642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:31.096642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:31.096642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:31.096642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:31.096642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 310/632] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.097073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b967-a6e7-4f4a-b8fc-85e3a48148ef None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.097158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b967-a6e7-4f4a-b8fc-85e3a48148ef None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.097415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b967-a6e7-4f4a-b8fc-85e3a48148ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.111214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 323/633] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:31.112514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.122608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8be7b967-a6e7-4f4a-b8fc-85e3a48148ef None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:31.123970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8be7b967-a6e7-4f4a-b8fc-85e3a48148ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:31.130439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.204593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 311/634] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:31.204593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.204593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.204593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.222412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:31.223632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:31.248284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565e7e-1b0e-4f1b-b192-553892ca9738 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.285015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.285659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.285979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.286258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.294070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.299520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.300919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.305383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 312/635] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:31.306635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:31.317099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.318028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 324/636] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.319518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.386495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.435376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da4ca-6f9d-4843-b3e7-e5752558347a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.449030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f2d7-641b-4e9d-8dcb-e24a38254024 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.471435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.472149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.472471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.472708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.473407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:31.473707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.475117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.484493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.485145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.485427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.485701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.487054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 313/637] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:31.487966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.492840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.494344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:31.506262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.508356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 325/638] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.509409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.558918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.611298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d149d-d0b6-4a30-bf25-2ebf9c315ee6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.619930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e243d-2f58-499a-a094-ca182bf7a242 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.640776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.655337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.656501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.656782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.657089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.657244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.657768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.658081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.658348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.659052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:31.659354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.660737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.669274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:31.673240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 326/639] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:31.674102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:31.674799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.675700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 314/640] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.678626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.786167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6577e-f8ce-4c4c-a64f-6f07761af567 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.798992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d85707-ba5c-449b-95bd-17e8d9e9afce None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.800347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.822585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.823228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.823546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.823773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.831001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:31.832438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:31.842073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:31.842884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:31.843044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:31.843264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:31.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.846411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 315/641] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.848042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.859860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:31.864861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:31.866219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 327/642] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:31.867343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:31.918887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:31.973714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b807b-c5e4-425f-bf1f-91be46b347fd None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:31.988825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42a3f-6a4a-4453-98ad-77d52837968f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.016337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.016962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.017241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.017518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.018293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.018600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.020043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.034801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.035237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 316/643] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.035462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.035745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.036021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.036116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.044930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.046460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:32.058913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:32.060104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 328/644] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:32.061018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.080959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.167980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124cce7-eaf9-4ccb-83dc-e17609aab23c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.170591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.174482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f9f-8feb-4fa5-bcfd-d46a7f79e66b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.206852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.207482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.207756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.208024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.208757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.209152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.209578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.210427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.210629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.210922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.211043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.211681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.211991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.213451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.221733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 317/645] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.222909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.229633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 329/646] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.230536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.329125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.335037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b372bc3-3e63-47f4-b64d-3c5d0701948a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.335590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3594e3-395a-4c52-9605-a4200ff5b0d9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.370057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.370676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.370957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.371223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.371938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.372100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.372225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.372728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.373009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.373281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.373612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.374699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.375012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.376448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.382933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 330/647] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.383222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 318/648] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.383772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.384011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.427990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.529298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633d178-a8a7-4490-bfdc-90393ff0bdd3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.535730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3841b3b-ba43-4435-ad26-7beaa2c323b3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.570661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.571320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.571603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.571870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.582288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.583173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.583609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.583874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.584005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.584909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:32.585022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.585332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.587256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.610778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 331/649] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:32.611698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:32.613203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:32.614262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:32.615158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:32.615158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:32.615158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:32.615158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:32.615371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 319/650] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:32.616696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.631007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.714276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.737343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67819-05b2-470d-95fd-b3fb1e974669 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.746566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0121139-9dbd-494a-9de4-a6e68e8f38f5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:32.782321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.783013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.783297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.783573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/59f92dd2b31f4a6dbb732ee67575ac1f/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.791491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:32.792428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:32.792720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:32.792998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:32.800409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=59f92dd2b31f4a6dbb732ee67575ac1f, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.801892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.811448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:32.813134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:32.826383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 332/651] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:32 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/59f92dd2b31f4a6dbb732ee67575ac1f/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:32.826999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:32.841901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:32.844055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:32.845667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:32.845667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:32.845667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:32.845667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:32.856030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 320/652] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:32.856030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:32.948888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:32.994926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd8e9c-8b56-452a-a068-6236ed868b1d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.004333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5e1d2-7108-472d-b382-04bc2844d7bb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.032158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.039507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.040668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.041175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.043352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.051225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.051909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.052342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.052765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.053708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:33.054366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.056191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.077020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 321/653] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:33.079606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.126174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.127824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.235692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05465965-c1af-4eba-947c-e0f9f97ee5ea None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.258386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 333/654] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:33.259501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.266219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.281797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.282447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.283120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.283398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.284103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:33.284406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.285790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.297595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.304324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 322/655] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:33.305205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.381003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cd89-a334-45a3-b932-5cbe20e3ee05 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.423368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.424088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.424374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.424650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.436812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.439734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.444067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02eac-caa8-4ab0-95d2-f44624eb3ed4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:33.457342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:33.459587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 334/656] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:33.461829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.487008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.487664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.487939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.488209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.501832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.504303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:33.524836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:33.526371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 323/657] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:33.527930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.579628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.585450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.598249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863909-82b4-4acf-957d-21e37a727d3f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.639409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.641087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.641595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.642134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.651286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.653105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:33.667127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:33.668114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 08:58:33.669033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:33.669569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8769cf-666d-414c-81f5-c3db30565b5a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.676283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 335/658] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:33.677493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.710713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.711362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.711645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.711922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.712750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:33.713060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.714969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.738663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 324/659] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:33.739508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.832222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362a7ab-e0d8-4b90-beac-c627bf6247f1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.858975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.870562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293081b3-e4e9-4fff-b49c-8f999a80b7e9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:33.880411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.881505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.882033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.882533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.883612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:33.884170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.885912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.906359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 336/660] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:33.907793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:33.914058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:33.914689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:33.914966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:33.915236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:33.916031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:33.916333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:33.917764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:33.931884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:33.949157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 325/661] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:33.949998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.050930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaed1-c482-46b7-861b-3dbe20023dca None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.064930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55077649-cc2e-4cfb-b502-71e898fdabb1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.099757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.101014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.101498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.102018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.119156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.120237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.120733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.121270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.131354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.132845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:34.147089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:34.148043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:34.148915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 326/662] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:34.149383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.166848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.204672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.206174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.222403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.298007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde079cd-a70a-4f15-be0d-19bdaf8d9576 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.340189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.341116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.341825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.342395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.353961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 337/663] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:34.355247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.443356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.444949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.458148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.498670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bf448-48c6-4bc3-a187-ba363ce4da8c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.516067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.540637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.541666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.542275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.542698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.543647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:34.544100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.546466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.556534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 327/664] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:34.557050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.566863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 338/665] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:34.568540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.674700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22864a0-f881-480b-b39e-6af0f8a053a3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.683294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58608b-60f3-4621-a2c3-ef67e5dc14c1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.687295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.717360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.718333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.718623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.718881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.723328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.724181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.724608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.725172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.729812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.731305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.745167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.746889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:34.747449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:34.748461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:34.750007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:34.750007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:34.750007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:34.750007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:34.750007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 328/666] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:34.750387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-04908073-a1cd-47f3-944b-b32b22edc966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:34.779509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:34.781289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:34.782406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:34.782406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:34.782406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:34.782406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:34.791342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 339/667] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:34.793583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5aef6-a3c5-406a-9818-125af5801704 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.794078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5aef6-a3c5-406a-9818-125af5801704 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.794698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5aef6-a3c5-406a-9818-125af5801704 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.801934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 340/668] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:34.801934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5aef6-a3c5-406a-9818-125af5801704 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.829157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.885262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a5d1-0e9c-41c7-8660-857069da51f3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.921065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5aef6-a3c5-406a-9818-125af5801704 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:34.929408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.930014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.930301 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.930575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.931636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=40d10d1a51524ba2967695566bb78e42) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:34.931944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.933460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:34.940214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 329/669] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/role_assignments?user.id=40d10d1a51524ba2967695566bb78e42&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:34.940868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:34.976592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:34.979998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:34.981272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:34.981748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:34.982410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:34.985048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:34.985757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:34.987484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf37f6-b16a-4e35-ba07-f1ee917c7e5b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.004238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 341/670] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:35.005773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a2c4-5c03-42c0-a292-eb4578ccd648 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.006214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a2c4-5c03-42c0-a292-eb4578ccd648 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.006582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a2c4-5c03-42c0-a292-eb4578ccd648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.027150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-45c4a2c4-5c03-42c0-a292-eb4578ccd648 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:35.028413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-45c4a2c4-5c03-42c0-a292-eb4578ccd648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:35.099447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed5a83-dd62-4fd2-aef4-e13a74859e9f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.158039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 342/671] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:35.159928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2740e-9a6c-44ff-b28a-fed2a37fa1c5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.160215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2740e-9a6c-44ff-b28a-fed2a37fa1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.160491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2740e-9a6c-44ff-b28a-fed2a37fa1c5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.162751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 343/672] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:35.163978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2740e-9a6c-44ff-b28a-fed2a37fa1c5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.170706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.171426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.171715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.171991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.172698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:35.173005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.174493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81abac-3c39-4cc6-9952-650562059d7c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.189988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.191773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 330/673] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:35.202751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.203814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.204103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.229866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:35.231083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:35.315453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 331/674] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:35.316425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.331898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2740e-9a6c-44ff-b28a-fed2a37fa1c5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.349958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.383622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.384669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.386050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.386333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.403498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.406268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:35.423725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:35.424662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:35.425536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:35.430087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 344/675] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:35.431751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.479132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330339-e382-4919-99cd-0346e9c90d05 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.519070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.519755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.520038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.520314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.526606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.531102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:35.536083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:35.537032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:35.537876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 332/676] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:35.538587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.563206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28bcaf-0429-448b-9dda-30626e7bf10e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.599328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.600321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.600774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.601203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.608689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.610583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.617370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:35.624641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:35.625521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:35.626426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 345/677] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:35.626897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.650272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eac87c-eea2-4c5b-803b-164f078274bf None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.688267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.688921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.689179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.689458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.699652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:35.706008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:35.706995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:35.707810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:35.712391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 333/678] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:35.712951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.739942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff62b3-c568-4ea1-a01e-537ae89d65a5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.776196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.776894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.777175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.777452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.778297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:35.778599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.780102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.781746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.797312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 346/679] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:35.797312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.827086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d3d89-5e73-4269-b9c1-e2796be2889b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.862134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.862760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.863041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.863313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.864030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:35.864334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.865713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.868774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:35.877515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 334/680] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:35.882979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:35.907836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b47bd-1d06-4367-81f0-3aa572d96ca1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:35.953581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:35.954357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:35.954634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:35.954957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:35.955644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:35.955948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:35.957385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:35.969971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 347/681] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:35.970773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.033704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653bf50-6220-4369-9a26-301ee4349771 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.043093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.084407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.085058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.085731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.085731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.086531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:36.086877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.088506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.097371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 335/682] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:36.098731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.102070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65697520-fc5c-47d1-89fd-56beea869e6f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.146423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.147044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.147594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.148390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e4e8151c2541400cb76b8b19384f49d8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:36.148694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.150147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.156853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 348/683] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:35 2022] GET /identity/v3/role_assignments?user.id=e4e8151c2541400cb76b8b19384f49d8&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:36.157683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.193934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.261830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd3c05-0493-4069-802f-9c4e3e23103c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.309562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.310443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.310820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.311193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.316758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622d58f-f659-4ffb-94f2-68fd380c8ac0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.325524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.327209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.344627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:36.346720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:36.348881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:36.350232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:36.350232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:36.350232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:36.350232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:36.351043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 336/684] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:36.352180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.365697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.366433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.366712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.366982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.367771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=6077e3f936ea42acbc6879758ddc8336) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:36.368070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.369524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.376414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 349/685] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/role_assignments?user.id=6077e3f936ea42acbc6879758ddc8336&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:36.377702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.484616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6d351-2bf3-4604-b293-9e5a38a731a5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.502558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.529760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.531150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.531498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.531847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.545699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.547981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:36.565690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:36.566655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:36.567534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:36.567534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:36.567534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:36.567534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:36.567534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 337/686] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:36.574423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.650605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f26d-281f-412e-8ac8-1d03d6e5762a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.697864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.698690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.699025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.699394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.700150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:36.700455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.701977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.715688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 350/687] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:36.715996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5326f0-d059-4647-8a89-511cf89aa8d9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.721815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.758418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.759052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.759332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.759607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.760453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:36.760900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.762522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.779392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 338/688] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:36.781059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.785744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.822840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:36.871418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120fe6-be3e-46fd-8d00-3e3725b732ee None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.906605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86a924-fb03-4e45-9225-d2f405f763f3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:36.924454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.925030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.925310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.925584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:36.946973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:36.951453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:36.966943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:36.968815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:36.969725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 351/689] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:36.975928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:36.983391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:36.984061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:36.984345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:36.984622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.000768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.002277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:37.020696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:37.021904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:37.023340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:37.025900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 339/690] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:37.030899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.203926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.223063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26048a1-772b-44d4-a6c4-e5525a3277bd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.224954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.236351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64711ea-297d-4a2d-b7f1-168c43942fb7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.275134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.276170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.276598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.277015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.278133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:37.278664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.280874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adcaac-569c-4887-b0e1-695971a710eb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.287593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.288358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.288741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.289144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.290316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:37.290739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.292338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878786b-2adb-448b-9fc9-acdb41ca87bb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.308963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 352/691] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:37.310539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926c16-842e-46f2-b930-16b3aec1bc7e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.310968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926c16-842e-46f2-b930-16b3aec1bc7e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.311389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926c16-842e-46f2-b930-16b3aec1bc7e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.313044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 353/692] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:37.314703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69926c16-842e-46f2-b930-16b3aec1bc7e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.319611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 340/693] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:37.328071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.328591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.328981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.349913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:37.351445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:37.428324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 341/694] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:37.429517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.445948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69926c16-842e-46f2-b930-16b3aec1bc7e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.492843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.493879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.494347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.494798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.510211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.536000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.548690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649ab2d-8bf0-4d61-b9b0-45afe275cc07 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.585266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.587214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.588662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.589365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.589651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.589961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.669925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.671588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.715130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 354/695] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 08:58:37.716432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.754552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.782604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:37.799170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 342/696] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 08:58:37.800146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.835337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116c1ad-4451-415c-9b26-48e13ad46b3b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.878176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.879233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.879712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.880374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.881213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.883542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:37.898085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:37.899132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:37.899981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:37.903142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 355/697] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:37.904101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:37.932357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d78c-ab1b-4aee-913f-78bd583a2be1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:37.978631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:37.979588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:37.979965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:37.980647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:37.994697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:37.996310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:38.017106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:38.018175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:38.019038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:38.019038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:38.019038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:38.019038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:38.019038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 343/698] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:38.019786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.029332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef45e87-4c63-4169-8eab-c2c1858a427c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.052841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.057089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.080646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.081315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.081595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.081888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.097315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.099053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:38.118484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:38.119453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:38.120325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:38.120325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:38.120325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:38.120325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:38.120325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 356/699] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:38.121043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.162326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1ce2-e520-4c54-ae39-e8a87a2d34a6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.214418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.215502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.216064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.216572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.217493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:38.218688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.221201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.227458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 344/700] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:38.228767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.258927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92222b1-4057-4698-9022-00cab398c77f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.306598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.307254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.307612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.307889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.308583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:38.308883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.310289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.326204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 357/701] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:38.327741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.374212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.377256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.394797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11541e9-47de-49b4-ba0b-f3ff6a40a33d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.440399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.441025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.441319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.441576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.442300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:38.442603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.444296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.462356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 345/702] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:38.463515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.467468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1184-dd8d-4423-b8d2-b788007d83f9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.507341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.508400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.508885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.509376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.510303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:38.510814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.512572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.520052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 358/703] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:38.521341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.626648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a921a8e-2cc2-4d34-b469-bc64d7bfe8e2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.677756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.678503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.679105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.679392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.679779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.683755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9e486-b8ba-46e9-a107-624338e76ba6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.690413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.692397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-bdce1174-43a8-48da-84be-02af5968e0b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:38.707506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:38.708438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:38.709304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:38.711937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 346/704] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:38.713439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808df4f-6d98-4140-b064-606906aa2c6f None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.722116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808df4f-6d98-4140-b064-606906aa2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.722522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808df4f-6d98-4140-b064-606906aa2c6f None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.723547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.724167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 347/705] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:38.725660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808df4f-6d98-4140-b064-606906aa2c6f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.728196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.728558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.728843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.738591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.738825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.741569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8924ca67-d5c1-4013-a52d-a2581773414e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:38.754469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:38.756272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 359/706] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:38.757437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.758653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.758653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.778028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:38.779253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:38.838920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808df4f-6d98-4140-b064-606906aa2c6f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:38.862015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 360/707] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:38.863127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.877220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:38.877767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:38.878078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:38.878361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:38.879118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:38.879423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:38.880867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:38.894596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 348/708] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:38.900457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:38.985020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:38.993134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:39.001208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3418e-f0d1-40b2-86a3-9ea1b8d61441 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.033751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6d54-9ee9-4a55-a061-8587b15e18d3 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.056216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.056847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.057122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.057395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.058487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:39.058828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.060323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.073880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 361/709] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:39.074131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.083509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.084161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.084485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.084823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.085500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:39.090381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.099253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08994f3-5b3e-4b62-b0ad-b8520f0d31ef None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 349/710] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:38 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7912a-1159-4cb2-9d4c-6ea61556e012 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7912a-1159-4cb2-9d4c-6ea61556e012 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7912a-1159-4cb2-9d4c-6ea61556e012 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 350/711] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:39.112887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7912a-1159-4cb2-9d4c-6ea61556e012 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.251801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f8bd-cdf6-46e8-916e-e9b8bafaad33 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.262601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7912a-1159-4cb2-9d4c-6ea61556e012 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.297360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.298140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.298427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.298702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.310792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.312715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.319698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:39.330436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.331135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.331418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.331692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-5440cead-b2c0-4c5a-b714-68bf21f94005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:39.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:39.332785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:39.334057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:39.334057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:39.334057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:39.334057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:39.334341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=59f92dd2b31f4a6dbb732ee67575ac1f) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:39.334341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.334508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 362/712] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:39.334603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.339288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.339288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.339288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.341864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 351/713] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/role_assignments?user.id=59f92dd2b31f4a6dbb732ee67575ac1f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:39.342809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.346525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:39.382168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:39.383400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:39.507645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 363/714] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:39.541792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.570741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d2e85-cf0c-458c-9dfa-6f54b74f8643 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.620909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.621550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.621828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.622144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.622852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:39.623158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.625163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.644269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 352/715] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:39.646662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.684340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc7e7-4333-4529-ab2f-1b14b27297e8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.725573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:39.727323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:39.732492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.733943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.734435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.734893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] PATH_INFO: `/v3/services/network` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.735705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.737254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:39.748345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:39.750162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:39.750991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:39.753894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 364/716] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:39.760984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:39.785333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97692d-7360-4586-95f7-35761f5fb6d8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.825446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.826111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.826394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.826671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.827438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.829012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.855083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 353/717] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:58:39 2022] POST /identity/v3/users => generated 254 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:58:39.914736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c6bda-87b4-477c-b264-4ce32301295a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:39.966686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:39.967326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:39.967679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:39.968014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:39.968868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] RBAC: Adding query filter params (name=network) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:39.969364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:39.971004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b651a-5622-4245-8310-9e0659bf0e53 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:39.976597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 365/718] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:39.982367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:40.139410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66472-80c6-4249-97c1-a6fab12a56b5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:40.157032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:40.193208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:40.194225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:40.194688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:40.195146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:40.196338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] RBAC: Adding query filter params (type=network) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:40.196887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:40.199399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b1b0-baf1-42cc-9490-b5a2b443bb0a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:40.207017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 354/719] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:40.856627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad125cf3-8881-4533-8f06-2cdfdb63202a None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:40.857186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad125cf3-8881-4533-8f06-2cdfdb63202a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:40.857186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad125cf3-8881-4533-8f06-2cdfdb63202a None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:40.858893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 366/720] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:40.864590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9faa7-67a2-4912-91c4-492247575955 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:40.865068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9faa7-67a2-4912-91c4-492247575955 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:40.865144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9faa7-67a2-4912-91c4-492247575955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:40.889030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8bb9faa7-67a2-4912-91c4-492247575955 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:40.890788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8bb9faa7-67a2-4912-91c4-492247575955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:41.013801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 355/721] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:41.046827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad125cf3-8881-4533-8f06-2cdfdb63202a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:41.155733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:41.210439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad125cf3-8881-4533-8f06-2cdfdb63202a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:41.260548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:41.261189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:41.261475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:41.261749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:41.262485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:41.263878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f653-b903-4dd5-b311-a26d5df8e470 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:41.299484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 367/722] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:58:41 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:41.744398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4595-befa-40a0-a36f-2e72fc6602c3 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:41.744693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4595-befa-40a0-a36f-2e72fc6602c3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:41.745001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4595-befa-40a0-a36f-2e72fc6602c3 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:41.746656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 356/723] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:41.752423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:41.752710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:41.752990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:41.773086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:41.774606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:41.876911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 368/724] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:41.913819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc4595-befa-40a0-a36f-2e72fc6602c3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.082319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc4595-befa-40a0-a36f-2e72fc6602c3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:42.095163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:42.130827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:42.131487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.131769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.132044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.153629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-7a912080-ab03-4dbb-a1a8-eea04620d9dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:42.159601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:42.160623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:42.161464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:42.163519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 357/725] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:42.169457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.246651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:42.250625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7da71-ca6b-4915-9888-8f58048e61b3 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.251184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7da71-ca6b-4915-9888-8f58048e61b3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.251690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7da71-ca6b-4915-9888-8f58048e61b3 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.253557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 358/726] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:42.261798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.262459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.262942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.298124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:42.299384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:42.332896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc951b75-c1aa-4ac4-ad9a-febf2eb2a7eb None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:42.397585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:42.398260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.398536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.398810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.399549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:42.399855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:42.401242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:42.423067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 369/727] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:42.426792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.444915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 359/728] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:42.503818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.613902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5e56-b3ad-495f-b6e8-7e7f9b5ffa28 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:42.632786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:42.669118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:42.671286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:42.671979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.672260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.672534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.676000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162dec4-9bac-48dc-bb81-128bd957c398 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:42.698695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:42.700154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:42.734004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:42.734932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:42.735800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:42.735800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:42.735800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:42.735800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:42.737032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 370/729] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:42.742899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.746578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:42.747259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.747539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.747813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.768190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-7f9ecbd4-6a4b-4165-9ef5-61ed7ce9f643 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:42.774266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:42.779793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 360/730] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:42.782154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f0ece-0b3e-4112-baa2-b816ae7e4696 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f0ece-0b3e-4112-baa2-b816ae7e4696 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f0ece-0b3e-4112-baa2-b816ae7e4696 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 361/731] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.786337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.809721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:42.811952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:42.884480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119afe-2539-4379-8ff8-98729d1463ef None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:42.905869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 362/732] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 08:58:42.907186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.937195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:42.937767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.938083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.938356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.939206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:42.939571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:42.940956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51522a08-136d-4018-b8bb-9bd386ebe092 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:42.954209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 371/733] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:42.959781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001348-245e-47a4-937f-2d908e178913 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:42.960159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001348-245e-47a4-937f-2d908e178913 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:42.960357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001348-245e-47a4-937f-2d908e178913 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:42.961724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 372/734] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:42.962642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7001348-245e-47a4-937f-2d908e178913 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:42.986469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.016865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.033312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00fd8-605b-4ec9-a309-bde1722db183 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.068639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.069424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.069874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.070335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.071466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:43.071960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.074336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.088827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 363/735] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:43.090319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.115427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7001348-245e-47a4-937f-2d908e178913 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.154709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.156000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.156283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.156575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.169195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-0af11198-a4c4-4717-9019-7dd550271669 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:43.174801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:43.175735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:43.176547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:43.182466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 373/736] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:43.182466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.182466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.182466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.199502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:43.200725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:43.210320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2951-40c9-40bf-a1ea-b2c3d5b74967 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.245962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.246621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.247370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.247655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.257068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.258570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:43.271231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:43.272317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:43.273240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:43.277193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 364/737] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:43.278258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.282203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 374/738] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 08:58:43.283327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.293617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.305539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.404907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843010ad-d677-4e89-9680-4d2e47f1c4bb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.424335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44daaf-ba42-4bbd-9c7c-dcada0ff5968 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.455692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.456667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.456951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.457229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.465991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.466623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.466907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.467180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.467923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:43.468253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.469486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.469602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.471255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.479114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 375/739] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:43.479944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:43.496902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:43.498064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:43.499987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:43.499987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:43.499987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:43.499987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:43.500324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 365/740] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:43.501230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.574437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.581854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.612572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b30a-4604-4753-8509-0e8c9be78528 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.629206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d63ed2-82e4-40e8-b0a2-dc8b15d48176 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.650279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.650913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.651194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.651469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.652287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:43.652597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.654073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.667994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.668535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.668813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.669082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.669775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.671751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.680059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 376/741] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:43.680059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.697900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 366/742] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:58:43 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:43.697900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.823139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676cfbf-4fa2-400b-bead-29981c5a1f3d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.867963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30516fae-735e-42f2-bdbd-d9bba8aeab0d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:43.878099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.889150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.890745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.891126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.891578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.895152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:43.904762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.907074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.916863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:43.917493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.917773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.918139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.918779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:43.919079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:43.920469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7f17797d-9c47-4d5e-947d-cc30727fd304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:43.933177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:43.934403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:43.935291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:43.935291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:43.935291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:43.935291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:43.935523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 367/743] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:43.935596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 377/744] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:43.935672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:43.936147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed9778-b051-4dce-9b24-c6645f96e15f None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:43.936259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed9778-b051-4dce-9b24-c6645f96e15f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:43.939759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed9778-b051-4dce-9b24-c6645f96e15f None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:43.939759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 378/745] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:43.939759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed9778-b051-4dce-9b24-c6645f96e15f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.058933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fcd46-14aa-4342-a1e9-6eb2f604a525 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.059502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed9778-b051-4dce-9b24-c6645f96e15f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.097791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.098488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.098833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.099031 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.099532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.101060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.101478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.101755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.107937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.110146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.115204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.116747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:44.123246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:44.124788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:44.125988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:44.129187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 379/746] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:44.130078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-28f49392-4ed9-4f99-b451-0be123e5fef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:44.136876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:44.137938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:44.139021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:44.139021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:44.139021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:44.139021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:44.145821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 368/747] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:44.145821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.145821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.145821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.167515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:44.168892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:44.188991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.196653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.247811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 369/748] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:44.248889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.265465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951be7a-ba5f-4cce-8f07-6051e9e19134 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.314197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.315253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.315826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.316248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.317160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:44.317761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.319628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.333510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 380/749] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:44.339346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.385084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac902f53-e3c3-4417-8387-f962307f56a7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.427627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.428203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.428485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.428766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.430118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:44.430444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.432923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.455460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 370/750] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:44.460873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.465221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3766149-443b-4dc2-b1e3-46001ef2092f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.466895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.485149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.517473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.518152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.518435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.518710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.519415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:44.519720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.521114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.534062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 381/751] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:44.534918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.605086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c195c-4b1b-429b-a646-e3efe342bc7b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.643627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.644274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.644559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.644830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.645647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.647083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:44.660828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:44.661823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:44.664144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 371/752] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:44.666029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.672008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cebd16-a8ab-4636-aaf1-74ac4931aa7f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.715270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.716242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.716720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.717170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.728316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.730708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:44.744474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:44.745645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:44.746753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 382/753] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:44.747330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.774909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.787356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:44.820048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ae673-a144-482c-8f6c-7e3c2df5199d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.877065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192f79-2111-431a-aa6b-04a3e10e96bc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:44.880504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.881553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.882046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.882518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.917167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:44.917829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:44.918144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:44.921455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:44.927211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.929975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:44.943546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:44.945140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 383/754] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:44.953303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:44.985938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:44.987476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.056510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.077704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163870-0344-479a-9308-3e3e7fb422fc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.083968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.116446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.117075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.117358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.117635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.118360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:45.118670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.123615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 372/755] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:45.124208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.124364 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.130513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 384/756] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:45.133882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.251070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4a53f-bde8-4c63-9cda-b08f73052ad8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.272807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f5429-d935-4253-bd83-f3330436991a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.292814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.293454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.293736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.294045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.294807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:45.295117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.296599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.313989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 385/757] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:45.314945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.325805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.333220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.334334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.335192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.335824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.337362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:45.337950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.341128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.374639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 373/758] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:45.375434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.423591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.456346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599940-93bc-4fd7-a112-40b053fcff9a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.489555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6a302-eee8-459e-8b67-b1507d60e8e4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.497530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.498764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.498764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.499598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.510700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.512402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:45.531706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:45.532682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:45.533560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:45.533560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:45.533560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:45.533560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:45.533560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 386/759] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:45.533829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.537898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.538536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.538932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.539215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.539906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:45.540243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.542030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.553688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 374/760] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:45.553688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.610379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.663646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e841-5057-4c60-94df-235e1c70e5a0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.689717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544bbe7-879d-4f84-ba60-69304aca33db None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.691708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.711878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.712804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.713231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.717456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.717456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=96a7013052fd4d48a2463cf51f13861f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.717456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.732039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 387/761] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:45 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:45.734249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.739083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.739755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.740045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.740321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.833447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.834975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3598a-74f6-434a-b6c1-695408ead0ec None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.872731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3211242-f2e1-49a3-b733-939d3f9af70f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:45.893433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:45.923686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:45.924542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:45.924542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:45.924990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:45.926298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:45.926298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:45.927385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:45.947970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 388/762] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:45.949135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:45.994918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 375/763] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:46.000230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d72d8-4051-408a-bb93-26d7d93a0ff3 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.000230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d72d8-4051-408a-bb93-26d7d93a0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.000230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d72d8-4051-408a-bb93-26d7d93a0ff3 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.001997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 376/764] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:46.003062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d72d8-4051-408a-bb93-26d7d93a0ff3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.006231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.071093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedc362-40b0-4e4f-b375-986d08fcfb46 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.121085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d72d8-4051-408a-bb93-26d7d93a0ff3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.124450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.125105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.125388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.125662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.126471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] RBAC: Adding query filter params (interface=public, service_id=f6e414fe3b38431f8f6b0cb8bc065420, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:46.126786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.128187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.149011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 389/765] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:58:45 2022] GET /identity/v3/endpoints?service_id=f6e414fe3b38431f8f6b0cb8bc065420&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:46.149011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.158025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.158682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.158965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.159240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.166736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.167971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-07f826f6-d009-4b26-8433-07ec6097bf28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:46.181402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:46.182374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:46.183242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 377/766] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:46.184304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.184304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.184479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.203877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:46.205081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:46.212455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.266175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.286415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 378/767] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:46.286415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.293823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723efe50-2b4d-4175-a5b8-b157525d816c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.336622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.337407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.337688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.337988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.351465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.352929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:46.370538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:46.371707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 390/768] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:46.379074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.411402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f2fe5-a8c6-45f3-b5d6-46fffc24989d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.450079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.450711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.450990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.451262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.452464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.454010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.459891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 379/769] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:46.466286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.504490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.517306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964970e-06b6-4a54-a9c4-009ae0218bbf None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.560039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.574643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.575720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.576532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.577495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.579229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:46.579751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.582041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.601565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 391/770] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:46.602971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.627858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078116-4a65-4b03-a7af-4d607eea9b47 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.683437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.684969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.685249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.685511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.686175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.687598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:46.697199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:46.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:46.699118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 380/771] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:46.699895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.768226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32100b0-66af-4732-ac20-3889631e057d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.812882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974704e-362d-479f-8910-068205f023d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.813707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974704e-362d-479f-8910-068205f023d0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.814265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974704e-362d-479f-8910-068205f023d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.814640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974704e-362d-479f-8910-068205f023d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.815510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974704e-362d-479f-8910-068205f023d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:46.815915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974704e-362d-479f-8910-068205f023d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.817413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974704e-362d-479f-8910-068205f023d0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.831067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 392/772] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:46.832201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974704e-362d-479f-8910-068205f023d0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:46.848137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f270-31fb-4bb9-a1eb-9385f1cbc5a5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:46.852978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.892080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:46.931493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:46.932142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:46.932421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:46.932694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:46.933513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=59f92dd2b31f4a6dbb732ee67575ac1f) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:46.933911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:46.935582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:46.943372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 381/773] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/role_assignments?user.id=59f92dd2b31f4a6dbb732ee67575ac1f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:46.944101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.008611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974704e-362d-479f-8910-068205f023d0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.074564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.076244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.077735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.078548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.079987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:47.081329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.083317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.089605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 393/774] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:47.096247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.149942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060323-e4a0-4d7a-8fef-08fd930367fb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.187369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.188027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.188306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.188583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.196079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.197542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:47.210790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:47.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:47.212712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 382/775] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:47.217826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.267003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:47.305788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:47.322389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314a6e-ed74-4d19-8d1d-4bfb52d053d7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.387518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.388158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.388439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.388714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.389437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:47.389750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.391176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.399563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 394/776] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:47.406112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.413071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218663d-a3d2-49b1-95dc-b7bc6b7f2df1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.491064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.492143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.492627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.493119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.494620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:47.495171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.496932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.517455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 383/777] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:47.523683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.586397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94a302-ef6f-4547-9602-c4de023ccd2f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.650633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.651419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.651702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.652908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.653696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] RBAC: Adding query filter params (interface=public, service_id=17e5dbf409f845589001a133eb0856c3, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:47.654034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.655989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.672473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 395/778] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:58:47 2022] GET /identity/v3/endpoints?service_id=17e5dbf409f845589001a133eb0856c3&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:47.682841 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:47.700217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:47.712089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498222f-8415-48c4-a82c-06773f24c616 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.722178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:47.779426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.780134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.780417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.780693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.781489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e0d7c7e659ec440786e599f1c0a64d48) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:47.781799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.783786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599b70f-6915-40a4-8218-b7d80ef831c4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.797031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 384/779] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:47 2022] GET /identity/v3/role_assignments?user.id=e0d7c7e659ec440786e599f1c0a64d48&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:47.861028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ff49-84bc-46b9-a233-dacf4b5467a2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:47.935903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:47.937919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:47.938882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:47.939375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:47.940272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:47.941941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33a078-c0cf-42b7-82e9-2148e077666e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:47.953953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 396/780] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:58:47 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:48.070334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48d8ac-60d2-4ac8-811f-320678e24253 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.070664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48d8ac-60d2-4ac8-811f-320678e24253 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.070883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48d8ac-60d2-4ac8-811f-320678e24253 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.072563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 385/781] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:48.078265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.078770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.078845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.098462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:48.099789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:48.182469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:48.188290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 397/782] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:48.220055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48d8ac-60d2-4ac8-811f-320678e24253 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:48.340433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48d8ac-60d2-4ac8-811f-320678e24253 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:48.378794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:48.379443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.379724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.379998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.391139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:48.396376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:48.397677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:48.399433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 386/783] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:48.407367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:48.545407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496abab-135e-4a67-a22c-ea8eca6f02b6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:48.583295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:48.583937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.584222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.584496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.585225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:48.585533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:48.586946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b17ac-1cbc-476e-902c-7433b8928ee0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:48.596693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 398/784] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:48.602253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:48.646861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:48.757796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba612-8774-4e03-8e94-6cb6b5eacd7e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:48.807032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:48.808053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.808631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.809132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.810512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344de738-681e-4b47-86bc-4eb304cb7a39 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.810866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344de738-681e-4b47-86bc-4eb304cb7a39 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.811147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344de738-681e-4b47-86bc-4eb304cb7a39 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.812870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 399/785] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:48.818676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:48.818982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:48.819227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:48.827938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:48.830975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:48.851380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:48.852582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:48.859023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:48.860699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:48.862546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:48.862546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:48.862546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:48.862546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:48.865516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 387/786] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:48.875287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:48.939801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:48.969892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 400/787] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:49.011058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.034245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.068237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb2d36-0b74-4355-8964-4114e7c7d855 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.159614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.160657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.161143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.161641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.162574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:49.163088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.164698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.182917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dbf8e-b233-4b52-8cba-a8f454eee603 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.193361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 388/788] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:49.198767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.224919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.226053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.227359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.229558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.231370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.233860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f6678298-ad21-4ee6-9be2-767b9327f458 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:49.251082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:49.252076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 08:58:49.252903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:49.257192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 401/789] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:49.258700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c488f5-ce23-456e-bcc2-378e78dcdf02 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.258986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c488f5-ce23-456e-bcc2-378e78dcdf02 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.259366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c488f5-ce23-456e-bcc2-378e78dcdf02 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.265134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 402/790] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:49.266029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c488f5-ce23-456e-bcc2-378e78dcdf02 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.338603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.347245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e7a5-539c-40c3-b7d7-55b98821b684 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.401140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.402161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.402450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.402726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.412029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.413920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.416604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c488f5-ce23-456e-bcc2-378e78dcdf02 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.417938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a1b90c97-72c8-49d8-9cca-a26eafe639f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:49.429829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:49.433152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:58:49.434119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:49.437470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 389/791] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:49.439202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.439791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.440268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.460887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:49.462543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:49.477823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.478538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.478817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.479087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.480084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:49.480399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.482373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.489251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 403/792] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:49.491033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.552248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 390/793] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:49.553343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.616724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c354aba-4f79-4cae-8c7e-28b89eaa89ef None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.643189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.658744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.659171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.659455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.659726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.660538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:49.661060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.662778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.685622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f07372-83e9-47d5-bd97-7d987a066739 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.689196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 404/794] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:49.694447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.720974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.729548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.730489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.730917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.731340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.732300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:49.733023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:49.735168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:49.746085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 391/795] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:49.747406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.809783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6f4e-26f6-466b-bb9b-5dbb088fbecc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.846934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.847588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.847866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.848137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.860584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:49.866097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:49.867321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:49.868487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:49.872219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 405/796] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:49.873239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:49.875116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cf88c-2bda-4544-a204-179914c06693 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.913178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:49.914145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:49.914574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:49.915012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:49.915557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.990593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205c3f-a3be-4bdc-b827-6b9344fef127 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:49.994216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:49.998830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.000835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.026503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.027133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.027417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.027690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.028407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.029778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.063973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 406/797] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:58:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:50.063973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.228529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 392/798] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:50.236345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.272838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:50.287967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7791b3-2750-41e5-ab11-787b6fb79629 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.325386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.326072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.326382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.326631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.327881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:50.328187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.329583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.339907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 407/799] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:50.346615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.395303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966f30b-b213-49cb-bfa4-be0454db385c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.399478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:50.442334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.443410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.443965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.444241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.458867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.460334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-baa52f9a-5dce-4f83-90e5-b9fd71383016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:50.476370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:50.478410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 393/800] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030877f-4381-483e-95e1-30ff38e87727 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030877f-4381-483e-95e1-30ff38e87727 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.480509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030877f-4381-483e-95e1-30ff38e87727 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.481421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 394/801] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:50.493867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030877f-4381-483e-95e1-30ff38e87727 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.502003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f2190-f696-481c-9cd1-133c5474f62e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.545084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.546271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.546775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.547147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.565438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.567367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-5581638a-f097-4134-9f31-108074afbb41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:50.586583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:50.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:50.588394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:50.588394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:50.588394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:50.588394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:50.588394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 408/802] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:50.589859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91d45-c5a2-4359-8995-f2ce3cedde73 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.593256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91d45-c5a2-4359-8995-f2ce3cedde73 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.593256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d91d45-c5a2-4359-8995-f2ce3cedde73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.601387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:50.618090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e5d91d45-c5a2-4359-8995-f2ce3cedde73 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:50.619594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e5d91d45-c5a2-4359-8995-f2ce3cedde73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:50.625158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030877f-4381-483e-95e1-30ff38e87727 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.663663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.664463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.664811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.665155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.665962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:50.666356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.668052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.682382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 395/803] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:50.683257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.695858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:50.712113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 409/804] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:50.713494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d51f-db2e-4eec-8fba-36471378ee76 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.713783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d51f-db2e-4eec-8fba-36471378ee76 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.714088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d51f-db2e-4eec-8fba-36471378ee76 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.719831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 410/805] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:50.719831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d51f-db2e-4eec-8fba-36471378ee76 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.826250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488c1a-7204-4558-bb8e-54c6532d9208 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.853128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d51f-db2e-4eec-8fba-36471378ee76 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:50.870628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.871346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.871630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.871904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.872597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:50.872907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.874851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bef817-d214-4cac-aad4-08d62bd17482 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.890245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 396/806] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:50.892808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.893176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.893561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.900233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:50.901152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:50.901509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:50.901957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:50.904280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:50.913704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:50.913940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:50.915223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:50.915357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:50.935697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:50.937462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 411/807] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:50.943777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:50.998790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.008402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 397/808] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:51.009085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.067707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498b28-144b-40c1-a574-0462003f22c5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.103611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.104308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.104558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.104949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.117375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:51.123661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:51.125512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 412/809] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:51.126414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.152263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9cd61-8c10-4d52-8ca8-ae08d275b7a9 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.188164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.194898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.195778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.196119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.196474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.205782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.207839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:51.220498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:51.221517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 398/810] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:51.223196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.253590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec20b-8695-44ae-b3a9-b9e867274bb2 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.284775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.293513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.295032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.295621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.296471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.297251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:51.297558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.299154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.317796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 413/811] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:51.319118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.350058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f6063-2b44-4326-bd30-728860355df2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.390743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.391329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.391857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.392143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] PATH_INFO: `/v3/services/network` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.393209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.394926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:51.407467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:51.408898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:51.409717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 399/812] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:51.410533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.448418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12af9a-5500-4e41-a6c0-8b7243acc728 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.474541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.497570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.498348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.498699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.498981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.499711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:51.500018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.501391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.514941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 414/813] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:51.514941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.532120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7536ed1-3412-43b4-a7fb-29a9d1ca142c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.557227 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.574663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.575295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.575580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.575856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.576662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:51.577005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.578498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.599285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 400/814] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:51.600151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.633250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7473a-7daf-4712-974e-6a14de972207 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.675674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.676597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.677069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.677535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.678943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:51.679452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.682275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.697882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 415/815] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:51.697882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.745782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96f673-18e3-4e95-bfbb-9db5c740ba69 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.786262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.798493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.799324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.799702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.800112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.800913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:51.801323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.802941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.809330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 401/816] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:51.810838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:51.858428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e0ea7-c25f-4a49-a4e4-b36f7cc2d5d9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.866564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:51.914313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:51.915183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:51.915548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:51.915900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:51.938938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:51.940438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:51.956558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d2409-4d32-4752-9d7e-64165ee3fcc7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:51.962645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:51.963557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:51.964492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:51.964492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:51.964492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:51.964492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:51.964492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 416/817] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:51.965631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.001997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.003293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.004189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.004940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.020491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.022507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:52.043147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:52.044243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:52.045281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:52.045281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:52.045281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:52.045281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:52.048215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 402/818] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:52.054277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.104299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.120022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b004-6a4e-419b-9b27-277a6900758f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.166156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0095-6e50-4caa-884a-b7104d9e3dd8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.170964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.171897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.172376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.172862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.173741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] RBAC: Adding query filter params (type=network) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:52.174205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.174464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.176081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.182099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 417/819] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:58:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:52.183166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.211054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.211667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.211956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.212233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.212946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:52.213255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.214828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.232345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 403/820] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:52.233386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.293017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2117c29-6f2a-490f-9539-27beabb1e83d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.330059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.331493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.331987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.332453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.333363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:52.333952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.335562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.346945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311de1a-9a6a-4fd3-bb10-b3af25ab7898 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.354192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 418/821] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:52.355443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.374879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.387730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.388350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.388696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.388975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.390336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.391958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.421946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 404/822] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:58:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:52.423971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.462579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.520622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c52768-97b4-4ac3-a5c1-66816b6ed275 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.577401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.578550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.578851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.579129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.588605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e81a23-a024-4176-ac9e-7a65e0fdb9c9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.589443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.592047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d41bad79-743d-4f24-af61-7338b0e4d1c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:52.609832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:52.610897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 419/823] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d87a6-8613-4e0c-89ab-837519788a53 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.617739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d87a6-8613-4e0c-89ab-837519788a53 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.619254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d87a6-8613-4e0c-89ab-837519788a53 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.619254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 420/824] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:52.619254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d87a6-8613-4e0c-89ab-837519788a53 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.648519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.649178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.649458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.649791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.662567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.664037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-684e5249-49f2-4c6c-a2fe-ef19232bf5d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:52.677562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:52.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:52.679702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:52.684755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 405/825] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:52.687077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.687077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.687373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.704750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.709876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:52.711551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:52.740739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d87a6-8613-4e0c-89ab-837519788a53 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.787700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.787938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.788523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.788813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.789254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.789937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:52.790249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.793640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:52.797482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 406/826] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:52.798828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.821565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 421/827] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:52.822538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:52.945319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be805a48-bd0c-4d8f-bd53-4273928ea32a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.956154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c992580-3f34-4b2a-996a-ee4bf2420456 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:52.989685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:52.990618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:52.991136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:52.991314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:52.991430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:52.992092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:52.992445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:52.993874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.000851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.002735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.003091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.003493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.005195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.007753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.010203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 407/828] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:53.012730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:53.019209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:53.020244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:53.026959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 422/829] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:53.027980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.084975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:53.134818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3656-540d-48f3-9365-1d5ee44788ae None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.148457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61b06b-8a25-4b96-a1ee-5daf0ed10beb None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.179116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.179773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.180054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.180598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.191737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.192668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.192831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.192974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.193220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/59f92dd2b31f4a6dbb732ee67575ac1f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.194324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.195251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=59f92dd2b31f4a6dbb732ee67575ac1f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.196691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:53.210351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:53.211316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:53.212245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:53.212245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:53.212245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:53.212245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:53.212245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 408/830] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:53.212849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.216231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 423/831] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:53 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/59f92dd2b31f4a6dbb732ee67575ac1f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:53.221096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.317882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:53.392715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c20f-29ad-42f0-bd5b-fa228537392e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.404778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898984-0d26-459f-a651-1d99e32c70ef None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.422610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:53.448676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.449663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.450293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.450876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.452151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:53.452686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.454074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.454960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.455245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.455655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.456119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.457360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:53.457898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.459984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.466187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 409/832] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:53.474350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.474989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 424/833] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:53.480167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.596316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adc636-e228-48a2-9f51-32eec7efa4ad None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.606202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866bc84-e651-48b3-8c8c-57cfd7b174d7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.622255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:53.633988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.634628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.634931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.635184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.636029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:53.636342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.637733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.649937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 410/834] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:53.652199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.652876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.653154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.653426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.665517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.667043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:53.680760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:53.681987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:53.683329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 425/835] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:53.691340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.743354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:53.810934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951a762-08fd-4ea9-8b3f-133d5b24ef53 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:53.848636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:53.849199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.849473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.849743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.850541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:53.850849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:53.852299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bdd8-dd8c-41d8-bbb2-8870726e439c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:53.867757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 411/836] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:53.873722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:53.954428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12af83-8359-446d-98f1-608b8ca25790 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.954773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12af83-8359-446d-98f1-608b8ca25790 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.954974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12af83-8359-446d-98f1-608b8ca25790 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.956774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 412/837] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:53.962573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:53.962847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:53.963128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:53.986081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:53.987303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:54.019515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd441d74-4a4d-4098-8744-fc1ded16a365 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:54.063828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:54.064518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:54.064802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:54.065582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e0d7c7e659ec440786e599f1c0a64d48/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:54.082813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 413/838] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:58:54.092058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=e0d7c7e659ec440786e599f1c0a64d48, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:54.094401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:54.106814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:54.118805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 426/839] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:58:53 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e0d7c7e659ec440786e599f1c0a64d48/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:58:54.162320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:54.281942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268db5d-2497-4708-be23-f5943aed4378 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:54.340046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:54.340926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:54.341268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:54.341613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:54.342294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:54.343895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:54.356020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:54.357231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:54.358211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 414/840] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 28 08:58:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:54.365167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:54.399523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:54.496173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8c5b4-d30c-44fd-806e-67a3c7203835 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:54.535689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:54.536588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:54.537059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:54.537480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:54.538359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:54.538816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:54.540367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec541951-9aa6-4448-b902-127e1e1f603e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:54.547122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 427/841] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:58:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:54.554491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:54.678379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:54.686273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64293a9-caa4-4f46-a6de-420374b380cf None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:54.725403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:54.726079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:54.726368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:54.726646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:54.727337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:54.727644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:54.729116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c4b72-e92f-49c2-944d-4bfc25ee73c5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:54.734656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 415/842] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:58:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:55.439925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fce4f-50bb-4776-8b9b-b49261d1faa7 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.440486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fce4f-50bb-4776-8b9b-b49261d1faa7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.440486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fce4f-50bb-4776-8b9b-b49261d1faa7 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:55.442188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 428/843] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:55.447823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.448120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.448402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:55.467949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:55.469161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:55.549779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 416/844] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:55.573395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fce4f-50bb-4776-8b9b-b49261d1faa7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:55.666139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:55.679893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fce4f-50bb-4776-8b9b-b49261d1faa7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:55.718926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:55.720329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.721025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.721721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:55.739593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:55.748602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:55.750771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:58:55.752202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:55.754960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 429/845] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:55.763276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:55.859237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:55.872296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53248a97-6dc2-458b-990b-72e1f99ee52b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:55.910016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:55.910652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.910935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.911210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:55.912279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:55.912593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:55.914023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb16978-84aa-41b9-9211-742a1b3a2e1e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:55.924653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 417/846] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:58:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:55.930328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:55.974828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad3f7-c82b-451b-a530-855a33791a48 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.975131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad3f7-c82b-451b-a530-855a33791a48 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.975397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ad3f7-c82b-451b-a530-855a33791a48 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:55.979363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 418/847] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:55.986769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:55.987261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:55.987711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.020160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:56.020160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:56.100289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116965d4-afa3-4998-a814-92c5a5528b1a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.112937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 419/848] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:58:56.143161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.158623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.159659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.160569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.161080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.176145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.178922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.190300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:56.197876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:56.198869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:56.200286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:56.200286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:56.200286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:56.200286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:56.200286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 430/849] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:56.206316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.260550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.290282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21255c-330e-4b2a-9214-358e75b778f2 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.324260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb646a4-313a-4e49-828d-c80d58fdf89c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.340911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.341549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.341853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.342149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.343479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.344874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f87138-dcb0-479b-a4a8-7ed335bbc609 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.361073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.365959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.368998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 420/850] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:58:56 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:58:56.369153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.369153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.369153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:56.369354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695804b-3278-4a65-bbca-9c0b1d843689 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.369423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.369507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695804b-3278-4a65-bbca-9c0b1d843689 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.369507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695804b-3278-4a65-bbca-9c0b1d843689 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.369632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.369700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 421/851] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:56.370700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.370987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.371265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.392675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:56.393454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 431/852] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:56.396116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:56.398909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.489888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.503469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 422/853] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:56 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:58:56.530299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.543899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080cab7-fe6e-47b5-b554-8c1ba2463c4a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.592793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.593450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.593731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.594145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.602791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.604486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.616263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:56.619608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:56.620701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 08:58:56.621787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:56.628420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 432/854] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:56.631064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.673250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9457edd-2edf-4cf6-9396-a259fd905e9f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.717911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.718873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.719293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.719705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.735018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:58:56.741017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:56.742319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 08:58:56.743387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:56.745788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 423/855] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:56.752150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457532-cdc4-45be-8efe-b9164c8e2b90 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.754808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.781128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.800500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.801146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.801428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.801708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.811993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:56.811993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.811993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.833884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 433/856] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:56.836886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.905710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93298-b26f-4881-b14c-8989f0a84362 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:56.941865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:56.953828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:56.954753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:56.955037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:56.955312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:56.956040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:56.956345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:56.958265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:56.973129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 424/857] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:56.975249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:56.977641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fada1-b258-403c-b784-b01bccd02648 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.019419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.020040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.020307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.020583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.084142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.095390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe35d7-cc97-4e12-904a-6754f0407665 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.112976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.114512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08d4cd-5790-4240-8b97-44d648018b4b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.137147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.137812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.138131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.138412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.153520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.155043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c359b2ff-e585-4a71-8558-ee7f99a284dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:57.175464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:57.176419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:57.177297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:57.177297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:57.177297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:57.177297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:57.178273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 425/858] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:57.180299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb47a7c-e38a-4468-9cc7-ba0ba86c25ce None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.180779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb47a7c-e38a-4468-9cc7-ba0ba86c25ce None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.181512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb47a7c-e38a-4468-9cc7-ba0ba86c25ce None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.184074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 426/859] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:58:57.189649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb47a7c-e38a-4468-9cc7-ba0ba86c25ce None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.239652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.260071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 434/860] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:56 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:58:57.261521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.261884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.262229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.281118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:57.282480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:57.325015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb47a7c-e38a-4468-9cc7-ba0ba86c25ce None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.380472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.381353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.381807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.390154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.390327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 435/861] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:57.390407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:57.390477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.390545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.390545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.403636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 427/862] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:57.405585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.406130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.523435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c10e5f-3ba8-4017-9147-62b3c21d8838 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.541511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.546497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5fb29-f509-4780-b486-7f4b48c94c6a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.579956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.581028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.581494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.581811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.589561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.590326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.590608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.590884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.595418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.596984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.602820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.604894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:57.614696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:57.619816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 436/863] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:57.621442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:58:57.622600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:57.629638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 428/864] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:57.634491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.697933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.758992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a218d0-68e8-48d5-8110-0179db35b2aa None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.764812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc986c68-8c24-4e8d-9cde-fb2829562f55 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:57.797732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.798381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.798661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.798941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.799623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:57.799930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.800086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:57.800691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:57.800970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:57.801241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:57.801344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.801967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:57.803328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:57.823856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 429/865] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:57.823856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.828449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 437/866] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:58:57 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:58:57.829730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:57.863679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:57.975010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526348a-2ff6-4a87-bb4a-06f87787c647 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.014126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a2a44-d4ac-4f57-a092-2ec27976a708 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.033947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.034960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.035246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.035521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.036059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.036322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:58.036723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.038360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.064327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 430/867] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:58.068801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.069385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.069669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.070684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.073826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.082775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.083095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:58.100388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:58.101543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:58.102825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 438/868] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:58.110762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.186919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.220731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec383-bacc-48ca-b976-cf623271fdf9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.246757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009f026-c719-4bb1-9747-3b56b7e0f151 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.277451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.278482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.278763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.279037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.291623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.292324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.292602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.292876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.293690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:58.294050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.295618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.314901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 439/869] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:58.316108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.371126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.381205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.382729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68212b4-61e5-4d7f-a867-415bd822c852 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.471276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df81f0-07f7-42f9-bf9b-9923dafb55b5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.486919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.526056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 431/870] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:58:58.530609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8ad8-d700-4c66-b9ce-af4c21f99996 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.530896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8ad8-d700-4c66-b9ce-af4c21f99996 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.531225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8ad8-d700-4c66-b9ce-af4c21f99996 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.532639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 432/871] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:58:58.533315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e8ad8-d700-4c66-b9ce-af4c21f99996 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.539496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.540140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.540424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.540700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.541505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=59f92dd2b31f4a6dbb732ee67575ac1f) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:58.541819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.543290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdaf616-2c38-459d-8b7e-90b0c6b2b0a9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.561873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 440/872] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/role_assignments?user.id=59f92dd2b31f4a6dbb732ee67575ac1f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:58.561873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.561873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.561873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.580446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:58:58.581939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:58:58.682275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 441/873] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:58:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:58:58.683725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e8ad8-d700-4c66-b9ce-af4c21f99996 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.697529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.727028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.727612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.727991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.728272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.731433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.742584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.744125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:58:58.761508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:58.762497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:58.763361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:58:58.763361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:58:58.763361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:58:58.763361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:58.763361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 433/874] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:58.768436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.817068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:58.836934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809ad1f-e138-4144-8eeb-971b2fb87eae None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.879400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.880063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.880344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.880613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.881255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.882671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:58.892523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:58.893588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:58:58.894529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 442/875] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:58:58.900315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:58.930979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f5ce-4d03-4449-a904-84ac530c275a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:58.971654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:58.972669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:58.973152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:58.973625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:58.974616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:58.975142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:58.977639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:58.991556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 434/876] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:58.996940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.029424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:59.038085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee257a94-32ef-41a3-a769-a524c7eb2098 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.083022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.084688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.085479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.086019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.087221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:59.088208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.090362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.098646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 443/877] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:59.104365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.138599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:59.153159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304442f-4380-485c-b71d-4c521c44dfe6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.192630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.193668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.194216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.194725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.203423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.205221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:58:59.218188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:58:59.219310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:58:59.220251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:58:59.222962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 435/878] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:58:59.229099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.232028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51983d-5bf0-45cf-a6b0-bb1d7b4f9304 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.275825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.277484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.277484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.277949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.279242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:59.279749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.282975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.296076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 444/879] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:58:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:59.297584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.336065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:59.368390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5ca98-8701-4137-bc45-74f4113f9b02 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.423265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.424269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.424690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.425195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.426703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:59.427273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.429379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.448644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 436/880] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:58:59.450013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.460014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf5df4-9a71-460b-a2d3-c92ca0be68d6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.465067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:59.519914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.520632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.521259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.521642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.523051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] RBAC: Adding query filter params (interface=public, service_id=bcf47a3f10dc4624a1bb8c009a5c1b70, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:59.523485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.525439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.540323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 445/881] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:58:59 2022] GET /identity/v3/endpoints?service_id=bcf47a3f10dc4624a1bb8c009a5c1b70&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:59.540323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:58:59.580556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06de90-489d-46ca-a5dd-de5accb3955b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.625177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.625814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.626162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.626489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.627230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e0d7c7e659ec440786e599f1c0a64d48) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:58:59.627538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.629015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c5087-b7b9-4c48-97aa-7c25a05f1383 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.641865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 437/882] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:58:59 2022] GET /identity/v3/role_assignments?user.id=e0d7c7e659ec440786e599f1c0a64d48&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:58:59.676294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:58:59.691795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d67c0-0cd1-471c-b0ec-b7abe2f3319e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:58:59.733078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:58:59.734274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:58:59.734556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:58:59.734831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:58:59.735511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:58:59.736990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416f3c7-5fe6-4c7d-b528-f9461eb8f6d0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:58:59.744143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 446/883] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:58:59 2022] GET /identity/v3/services => generated 1268 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:00.115283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391542-d201-4e05-a810-2602b41bcac0 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.115802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391542-d201-4e05-a810-2602b41bcac0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.115802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391542-d201-4e05-a810-2602b41bcac0 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.117452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 438/884] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:00.124591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.124983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.125234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.144843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:00.146179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:00.221113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 447/885] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:00.244382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391542-d201-4e05-a810-2602b41bcac0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:00.346366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:00.379509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391542-d201-4e05-a810-2602b41bcac0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:00.417705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:00.418402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.418687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.418966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.419613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:00.421043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:00.431538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:00.432603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:00.433496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 439/886] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:00.441248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:00.531520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:00.555886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c78d7-5b3e-457d-aa62-0e80fccd77b4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:00.595127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:00.596154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.596651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.597133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.598207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:00.598728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:00.600310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:00.606186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 448/887] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:00.612334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:00.731850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e776-c42d-49a1-ad09-f2270b6acbc2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:00.773895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:00.774923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.775408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.775884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.776778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:00.777285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:00.778996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daf5e9-b151-4cf2-8880-f5ebb9b54f4c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:00.790040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 440/888] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:00.792153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:00.835329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571e051-bf17-441e-9301-121a99a790c8 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.836900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571e051-bf17-441e-9301-121a99a790c8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.836900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571e051-bf17-441e-9301-121a99a790c8 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.838387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 441/889] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:00.848272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.848904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.849394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.856786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:00.871596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:00.873153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:00.929196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a27fb-bff8-4422-b211-3b9efb3b0b45 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:00.970018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:00.971063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:00.971548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:00.972023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:00.973003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] RBAC: Adding query filter params (interface=public, service_id=42bf4e290a894409bddf206eeb25cd77, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:00.973514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:00.974358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 442/890] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:00.982934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:00.993960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 449/891] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/endpoints?service_id=42bf4e290a894409bddf206eeb25cd77&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:00.997242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:00.997442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:01.012651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:01.126130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a579dd7-ab81-4dc9-b2ee-706e7cc4b91b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:01.153199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f67e66-ebe2-4e58-8aff-14126f138f47 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:01.158627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:01.165504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:01.166752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.167280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.167758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.168614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:01.170329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:01.184073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 443/892] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:01.195337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:01.196012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.196288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.196558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.197203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:01.198639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:01.209584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:01.210674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:01.211942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 450/893] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:01.217923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:01.344366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508998-df8a-4aac-b31a-6bd519c39b93 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:01.382263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:01.382820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.383099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.383372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.384054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:01.384357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:01.385734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562d0b-0d0d-4fc5-a3bc-3cebed23e667 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:01.391789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 444/894] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:01.398393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:01.489250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:01.534374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e2bc-c1b6-4170-ab22-ec1586aedcfc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:01.582602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:01.584525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.585335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.586050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.587036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:01.587559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:01.589163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50989b7d-d814-410d-8fc9-6ba95d30cfd2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:01.598428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 451/895] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:01.864586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1875-a638-43af-b6d4-62fde9dc2761 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.865291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1875-a638-43af-b6d4-62fde9dc2761 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.865828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1875-a638-43af-b6d4-62fde9dc2761 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.867783 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 445/896] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:01.873802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.874548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.875047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.894914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:01.896530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:01.924770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1ff8-b2ab-4ff4-b7a2-faacd112ffc4 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.925958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1ff8-b2ab-4ff4-b7a2-faacd112ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.926462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1ff8-b2ab-4ff4-b7a2-faacd112ffc4 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.928316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 446/897] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:01.933792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:01.934114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:01.934395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:01.965051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:01.966841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:01.990354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 452/898] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:02.017573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.044796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.060481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 447/899] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:02.097909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.121278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.162431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9f90-6516-4d64-a7dc-5d05bae847d0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.198410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.199074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.199366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.199650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.200294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.201683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:02.212049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:02.213131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:02.214001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 453/900] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:02.221954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.255038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113d170-8ce6-45c3-9b6b-979d41e94f2e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.296326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.297435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.298005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.298483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.310879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:02.316928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:02.317938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:02.318811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:02.321693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 448/901] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:02.327818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.340392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0045461-3a7b-4dad-a475-b7219d9428dd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.373672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.385497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.385708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.385974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.386267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.386970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:02.387279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.388694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.399876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 454/902] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:02.401181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.414272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.474655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7623c-8cf3-411b-8187-39732969ef40 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.519578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.520600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.521566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.522088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.523476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:02.524260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.526176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.538723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 449/903] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:02.546413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.555817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae955-a804-4ac4-9c01-139f5852dfd7 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.594977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.595831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.596011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.596235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.596924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:02.597224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.598762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.604567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 455/904] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:02.610690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.694726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.704160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721d51d-01ec-4101-91d4-49fbb4afa11a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.738223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:02.747974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.749184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.749872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.750210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.753356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963bc18-c2f7-4d2c-bbca-8947af1b40a1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:02.764121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.766459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:02.789058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:02.790051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:02.790936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:02.790936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:02.790936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:02.790936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:02.792164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 450/905] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:02.797668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:02.797908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:02.798909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:02.799465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:02.799941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:02.800939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:02.802591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:02.830226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 456/906] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:02.973332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4c9e4-e24f-4cdd-bfcf-5781627ab70b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.021452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.022164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.022458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.022726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.023434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:03.023794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.025249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.039052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 451/907] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:03.047405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.160872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf47d3-544d-4067-b05b-d8e29f25a2c1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.176349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:03.200379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.201047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.201328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.201605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.210309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.211771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-ab86a706-9a49-4c19-b1c5-7e942e9feacf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:03.224416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:03.226182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:03.227910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 457/908] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:03.235329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.262089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:03.349921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b988-c32e-4451-947f-5470d92816ee None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.379698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955969-0049-4df9-9f52-3dff59cc1dc0 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.380035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955969-0049-4df9-9f52-3dff59cc1dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.380253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955969-0049-4df9-9f52-3dff59cc1dc0 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.386636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 458/909] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:03.394150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.394150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.394438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.398485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.399137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.399423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.399702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.400528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:03.400842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.402355 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.425014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 452/910] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:03.430040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:03.431220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:03.435839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.514075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 459/911] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:03.531300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.540290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:03.572480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333e94-568d-4fae-b00c-6bd87d98b0d9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.618960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.620236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.621361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.621984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.661330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926b90-6c74-4100-967a-ad6db01e5d11 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.664769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:03.716605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4c943-f069-4b98-967f-127894312052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.717411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4c943-f069-4b98-967f-127894312052 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.717411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4c943-f069-4b98-967f-127894312052 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.718639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4c943-f069-4b98-967f-127894312052 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.719585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4c943-f069-4b98-967f-127894312052 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.720679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4c943-f069-4b98-967f-127894312052 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.722165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.724736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109416c7-188d-4373-98f5-c74384c0b32b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-57b4c943-f069-4b98-967f-127894312052 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:03.730929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:03.731900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 08:59:03.732720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:03.737468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 460/912] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:03.739236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4c943-f069-4b98-967f-127894312052 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.832945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:03.869895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 453/913] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:03.873938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74454668-1606-4db2-aaa1-368758d2af7b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.874423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74454668-1606-4db2-aaa1-368758d2af7b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.874887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74454668-1606-4db2-aaa1-368758d2af7b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.876505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 454/914] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:03.877728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4c943-f069-4b98-967f-127894312052 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:03.877967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74454668-1606-4db2-aaa1-368758d2af7b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:03.935420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:03.936439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.936897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.937339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.938450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:03.938947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:03.941556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e45ab-de22-422f-9298-1a725a4c0fee None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:03.955284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 461/915] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:03.955284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:03.955284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:03.955284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:03.981913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:03.990664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:04.005961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74454668-1606-4db2-aaa1-368758d2af7b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.020444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:04.055931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.056878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.057166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.057441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.075218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.075218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.082026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 462/916] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:04.083362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:04.093777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:04.094737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:04.095601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:04.095601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:04.095601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:04.095601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:04.099323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 455/917] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:04.100448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.136388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:04.215731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c1c6c-df6f-4045-8fcc-e13180dae0a3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.225021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfe9f-8f7b-42de-a67d-f769c1df6a2e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.267158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.267897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.268181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.268500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.269145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:04.269456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.269855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.270591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.270876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.270969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.271280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.272578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.274800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.278397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 463/918] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:04.279438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.313895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 456/919] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:04 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:04.313895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.322196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:04.427625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5267c-bf7f-47bc-92cb-729ca6d3689b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.461079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:04.481594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.482606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.482896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329dd34-6ef8-4034-bbf6-0827474150f2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.483645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.484125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.485691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:04.486425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.488493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.506832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 464/920] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:04.513104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.538312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.538547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.538794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.539224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.539686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.541097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.581980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 457/921] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:04.632090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:04.660269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b69ff-5849-41c1-a7f5-f64af9794558 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.707796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.709198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.709664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.710146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.719416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.721228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:04.735064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:04.736088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:04.736987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:04.739595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 465/922] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:04.749924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:04.872842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1d9a-f6b4-4549-a3ee-de08f3640325 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:04.909789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:04.910456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:04.910735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:04.911010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:04.911780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:04.912087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:04.913704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d102-64a0-4a91-9a25-6760ee3ea7d9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:04.928641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 458/923] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:04.934953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:05.026511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbacc0a-6351-476b-a117-ca01ef26d70c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.026760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbacc0a-6351-476b-a117-ca01ef26d70c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.027041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbacc0a-6351-476b-a117-ca01ef26d70c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.029219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 459/924] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:05.036052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.036694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.037038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.046388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:05.050268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09145379-5f11-49d6-8d89-0f6f2f6b945e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:05.062633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:05.063859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:05.104713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:05.105353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.105637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.105939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.106750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] RBAC: Adding query filter params (role.id=83477ffa00a4409bb4929e9c4b383c8f, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e0d7c7e659ec440786e599f1c0a64d48) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:05.107059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:05.109107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:05.117025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 466/925] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:04 2022] GET /identity/v3/role_assignments?user.id=e0d7c7e659ec440786e599f1c0a64d48&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=83477ffa00a4409bb4929e9c4b383c8f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:05.148168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 460/926] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:05.171679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:05.176736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:05.301156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e362e9-e230-4295-a5ee-5701fcf71f42 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:05.345223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:05.345880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.346170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.346450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.347140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:05.348602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:05.351335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING keystone.server.flask.common [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:59:05.373675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.sql.core [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:59:05.373675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:59:05.373675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [parameters: {'id': '91f33345ce9640d89a559084f9d87421', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:59:05.373675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80375) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f33345ce9640d89a559084f9d87421.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f33345ce9640d89a559084f9d87421. Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:59:05.391932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application result.read() Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 08:59:05.392814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application yield resource Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise self.value Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application session.commit() Jul 28 08:59:05.394295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:59:05.395317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:59:05.396305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application result.read() Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:59:05.397069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [parameters: {'id': '91f33345ce9640d89a559084f9d87421', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:05.398197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:05.398967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 91f33345ce9640d89a559084f9d87421. Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:05.399829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 467/927] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Jul 28 08:59:05 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:05.400370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:05.513766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916fd2e-c8dd-453d-bd3f-96c83b2ddd5b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:05.547805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:05.551638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:05.552289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.552567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.552840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.561999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:05.563475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-27eae98d-a5c4-4e8d-99a6-834b2a7860a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:05.575890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:05.577579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:05.578544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 461/928] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:05.583780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:05.657793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:05.724948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd692-2aec-4db5-b123-5a771077eb2c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:05.768316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:05.768952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:05.769231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:05.769503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:05.770264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:05.770568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:05.772331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d3ee-48c4-46e9-a060-61a296cc00c1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:05.795178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 468/929] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:06.529143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681974-5e1e-44ab-8c35-63777c96c0f5 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:06.529921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681974-5e1e-44ab-8c35-63777c96c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:06.529921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681974-5e1e-44ab-8c35-63777c96c0f5 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:06.531761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 462/930] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:06.537585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:06.538230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:06.538733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:06.559335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:06.561741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:06.638344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 469/931] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:06.661477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df681974-5e1e-44ab-8c35-63777c96c0f5 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:06.797264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df681974-5e1e-44ab-8c35-63777c96c0f5 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:06.843867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:06.844928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:06.845399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:06.845887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:06.846930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:06.849265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-45cf6412-6257-41ac-99b8-bca55c8a9be5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:06.863106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:06.864144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:06.864980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 463/932] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:06.872428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:06.927148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:06.949185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56305aae-4e8f-4e54-9e69-c0e0994d4a16 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:06.949517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56305aae-4e8f-4e54-9e69-c0e0994d4a16 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:06.949748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56305aae-4e8f-4e54-9e69-c0e0994d4a16 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:06.951894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 464/933] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:06.957170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:06.957462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:06.957748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:06.983773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:06.985051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:06.985447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809cc5a-20aa-4abc-9057-99a0a4faa618 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.025972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.026998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.027612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.028090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.029047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:07.029563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.031786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.042612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 470/934] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:07.043864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.046909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.073757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 465/935] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:07.098979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.193516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13410bd3-cbc8-4bbf-b4f9-03246893bbf3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.211820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.212794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aede37-7f22-410d-a4ca-5daca582f640 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.236665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.237303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.237586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.237879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.238581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:07.238890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.240279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.245715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 471/936] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:07.259817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.262644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.263481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.263865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.264811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.265594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.267759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:07.280073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:07.281356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:07.282254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:07.285947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 466/937] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:07.289741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.355347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.395354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabc58a-d434-497a-a41b-f3baac5d2b71 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.422045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f497e-938f-4e16-8dfb-ba93cbcf5b37 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.433444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.434483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.434964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.435428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.437293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] RBAC: Adding query filter params (interface=public, service_id=ed58f434f888496d84145cf327c79f42, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:07.437817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.440198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113fdf1-a95e-41ac-8e09-8a6fc52ca383 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.448915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 472/938] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/endpoints?service_id=ed58f434f888496d84145cf327c79f42&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:07.450297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870350f-0d0a-40d9-b4e2-a52e3b04f0e5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.450589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870350f-0d0a-40d9-b4e2-a52e3b04f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.450868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870350f-0d0a-40d9-b4e2-a52e3b04f0e5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.452321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 473/939] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:07.455259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870350f-0d0a-40d9-b4e2-a52e3b04f0e5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.468211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.468848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.469134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.469591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.470722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:07.471203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.473446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35459d-72f8-4226-bbc7-b22463f390bf None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.487174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 467/940] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:07.487174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849771-6776-4240-9756-8fba3de13936 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.487174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849771-6776-4240-9756-8fba3de13936 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.487174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849771-6776-4240-9756-8fba3de13936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.508313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.510165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6b849771-6776-4240-9756-8fba3de13936 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:07.511483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6b849771-6776-4240-9756-8fba3de13936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:07.593244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870350f-0d0a-40d9-b4e2-a52e3b04f0e5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.597760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 468/941] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:07.599257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b849771-6776-4240-9756-8fba3de13936 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.647549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.648426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.648703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.649230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.649951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.651326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.658511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 474/942] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:07.659657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:07.674057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.747962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b849771-6776-4240-9756-8fba3de13936 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.795158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816775f-4957-48b4-977b-440ffcf48bc6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:07.799526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:07.800552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.801712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.801712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.801712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.802630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:07.803319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:07.805051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:07.818637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 469/943] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:07.843515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:07.844623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:07.845133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:07.845412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:07.859092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:07.864351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:07.865336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:07.866248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 475/944] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:07.872315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.000813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8a7f-df87-4092-94dc-9823abd514cc None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.011957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.042274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.042915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.043211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.043512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.044211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:08.044517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.046650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4a67e-90db-4173-8a8f-4d869a86c990 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.057566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 470/945] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:08.063526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.093873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3af6a-3f19-4c86-818f-92055b23785d None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.094212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3af6a-3f19-4c86-818f-92055b23785d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.094462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3af6a-3f19-4c86-818f-92055b23785d None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.096146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 471/946] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:08.102203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.102696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.103245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.125371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:08.126677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:08.190285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d233f-c050-4dd0-8c25-7cadf74087d5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.218372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 472/947] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:08.236040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.237086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.237573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.238096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.247215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.254069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.255621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:08.283032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:08.284011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:08.284879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:08.284879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:08.284879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:08.284879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:08.284879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 476/948] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:08.291266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.311836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.372243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.375938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fc6be-e76a-4d9f-af5c-d1bddac91a4c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.419577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.420720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.421212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.421693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.422663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.424392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.427270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING keystone.server.flask.common [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 08:59:08.430253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894795-e169-462c-83c1-f13372de39e9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.442937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.sql.core [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:59:08.442937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:59:08.442937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [parameters: {'user_id': '244cab91c6664670aa253ed6528bb4ab', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:59:08.442937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80374) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result.read() Jul 28 08:59:08.453453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:59:08.454514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application yield resource Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application next(self.gen) Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise self.value Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application session.commit() Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 08:59:08.455683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.session.flush() Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application compat.raise_( Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 08:59:08.456606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 08:59:08.457465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application conn.query(q) Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application result.read() Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 08:59:08.459486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [parameters: {'user_id': '244cab91c6664670aa253ed6528bb4ab', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:08.461557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 08:59:08.463548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 08:59:08.465772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.469079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 473/949] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Jul 28 08:59:08 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:08.475502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.479699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.480647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.480929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.481202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.482980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:08.483297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.484683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.508344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 477/950] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:08.516075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.630016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.641333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce427cb8-e32f-4d4b-80fd-bda4a780c211 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.660215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924b3f6-e0fd-4984-a9e6-13936cbe25c1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.683079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.683884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.684368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.684746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.694202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.696096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.707273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.708188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.708639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.709102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:08.710565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:08.711742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 08:59:08.712971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:08.715076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 474/951] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:08.721150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.721801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.723428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:08.741092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:08.742350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:08.743281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:08.744612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 478/952] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:08.748866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:08.754439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.840426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55850f37-e47a-43ce-bae1-55142e5c411a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.878922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.879610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.879889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.880243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.881195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:08.881546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.883215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.906477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70856c-d310-419d-860a-96d9e056a166 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:08.946347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 475/953] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:08.955274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:08.955933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:08.956218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:08.956497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:08.957341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:08.957655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:08.959306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:08.981569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 479/954] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:08.982221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:08.989365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:09.090953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:09.113616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3555bf8-9167-4116-be78-064e28bb9b17 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:09.153478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:09.154605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.155629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.156073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.168676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:09.170174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:09.187445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:09.188421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:09.189285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:09.189285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:09.189285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:09.189285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:09.189285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 476/955] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:09.197311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:09.318766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45970-4b8d-4ad4-a28e-beebaea6c583 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:09.364583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:09.365238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.365557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.365898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.366643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:09.366986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:09.368402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:09.381041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 480/956] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:09.386895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:09.500499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba1927-a099-4a87-bb82-459f7f232a2b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:09.531971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:09.542031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11777326-2cad-4497-9c10-153e367023fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:09.543195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11777326-2cad-4497-9c10-153e367023fd None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.543482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11777326-2cad-4497-9c10-153e367023fd None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.543759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11777326-2cad-4497-9c10-153e367023fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.551495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11777326-2cad-4497-9c10-153e367023fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:09.553827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11777326-2cad-4497-9c10-153e367023fd None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-11777326-2cad-4497-9c10-153e367023fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:09.569718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:09.570709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:09.571587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 477/957] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:09.578901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:09.646488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:09.699642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aac86-c16f-4914-9137-514e3ce9e5ed None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:09.742764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:09.743405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.743687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.743962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.744720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:09.745027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:09.746585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ccc4b-ffac-43d9-af8a-7a94cbc18d8d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:09.765952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 481/958] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:09.768106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11777326-2cad-4497-9c10-153e367023fd None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:09.910426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11777326-2cad-4497-9c10-153e367023fd None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:09.939453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c647a-c0c8-42de-a5ed-9806fb336262 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.939751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c647a-c0c8-42de-a5ed-9806fb336262 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.940012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c647a-c0c8-42de-a5ed-9806fb336262 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.941739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 482/959] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:09.946636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.946922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.947196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.970061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:09.971271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:09.974753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:09.975608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:09.975898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:09.976195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e0d7c7e659ec440786e599f1c0a64d48/roles/83477ffa00a4409bb4929e9c4b383c8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:09.978541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=e0d7c7e659ec440786e599f1c0a64d48, role_id=83477ffa00a4409bb4929e9c4b383c8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:09.979995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369fd33-9658-4769-a11d-0e3095991b5c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:09.996374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 478/960] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:09 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/e0d7c7e659ec440786e599f1c0a64d48/roles/83477ffa00a4409bb4929e9c4b383c8f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 08:59:10.000936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689039-96c0-4d5c-8cac-fbf92d6afd18 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.001262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689039-96c0-4d5c-8cac-fbf92d6afd18 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.001492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689039-96c0-4d5c-8cac-fbf92d6afd18 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.002955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 479/961] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:10.012773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.013132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.013329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.034208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:10.036295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:10.059035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 483/962] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:10.083614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:10.108937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:10.145935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 480/963] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:10.168197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:10.174076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:10.254674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72499211-69d0-415f-9a96-8f5ae29196cc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:10.291099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:10.291760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.292045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.292318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.292946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:10.294368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:10.304782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:10.306226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:10.307376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:10.308127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0fa8c-5a76-4d03-95f4-75185ebb5ec7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:10.311303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 484/964] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:10.312046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:10.346237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:10.346869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.347150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.347425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.348120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:10.349522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:10.366704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 481/965] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:10 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:10.441441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704333d-74a7-4f1c-b946-63ddd65098d8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:10.452861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:10.493346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:10.494248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.494532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.494807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.495520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:10.495911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:10.497408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:10.506131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 485/966] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:10.511821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:10.577858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:10.644233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feae357-0a81-493b-a33c-1e29b5e8f05d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:10.690539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:10.691172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.691456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.691733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.692429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:10.692746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:10.694173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d96f1-4515-4a99-add1-a07a7f2e0b3e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:10.699642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 482/967] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:10.706216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:10.818457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e630a-8a7c-4f03-b07a-a00f82bba35b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:10.852405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:10.852941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:10.853218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:10.853488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:10.854226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:10.855590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07943a58-97c8-4ce4-9647-a099ecbecdce None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:10.878573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 486/968] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:12.078376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb488-cf21-4773-bbad-0e1d47db1666 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.078904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb488-cf21-4773-bbad-0e1d47db1666 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.078904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eb488-cf21-4773-bbad-0e1d47db1666 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.080480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 483/969] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:12.085982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.086484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.086562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.105752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:12.106982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:12.126977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:12.192481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 487/970] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:12.215952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eb488-cf21-4773-bbad-0e1d47db1666 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:12.342257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eb488-cf21-4773-bbad-0e1d47db1666 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:12.351139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:12.387461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:12.388105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.388382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.388652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.400790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:12.405820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:12.406885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:12.412886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 484/971] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:12.413589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:12.531268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c125e5c-a323-49ad-aebe-8743e1f7477e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:12.573225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:12.573877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.574167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.574447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.575194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:12.575492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:12.577420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae318f-7245-40b5-84a0-291c18e45daf None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:12.593518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 488/972] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:12.595114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:12.719881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d104-d481-4b78-ae2a-795dacecd5c8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:12.738027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225040-6ef9-47de-ab71-c6b07b53f884 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.738427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225040-6ef9-47de-ab71-c6b07b53f884 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.738768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225040-6ef9-47de-ab71-c6b07b53f884 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.740395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 489/973] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:12.745819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.746214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.746489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.791466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:12.791466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:12.791466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:12.791466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:12.797213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:12.798485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:12.807428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:12.813771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:12.817602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:12.837265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:12.838367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:12.839406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:12.839406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:12.839406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:12.839406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:12.839406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 485/974] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:12.844759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:12.877051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:12.896254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 490/975] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:12.927742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:12.989248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a9f6b-a266-4046-925f-8faa96ce97ee None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.025059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.025678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.025983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.026250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.026947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.027244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.028635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8a5e-d2ba-47e8-9881-65386a9300ef None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.041397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 486/976] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:13.042769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4ae7-e28b-4263-8b9e-f3996a99ee2a None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.042991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4ae7-e28b-4263-8b9e-f3996a99ee2a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.043266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4ae7-e28b-4263-8b9e-f3996a99ee2a None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.044626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 487/977] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:13.047658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c4ae7-e28b-4263-8b9e-f3996a99ee2a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.056349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbeb5b3-fe3b-4d04-bdbd-dc53631ac3b5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.093599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.094294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.094551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.094822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.095463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.096858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f2ad3eec-69ea-4844-bf18-242d51d16e8c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:13.110353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:13.112377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:13.114105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 491/978] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:13.115722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.115722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.115722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.127273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.133219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:13.134453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:13.150098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.176330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c4ae7-e28b-4263-8b9e-f3996a99ee2a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.212728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 492/979] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:13.213607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.245953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.246907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.247331 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.247765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.260984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.263949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:13.288818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:13.290252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 488/980] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:13.291707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.365636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011249e7-adac-43d3-bbaf-3cac1114da12 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.410219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.410905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.411230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.411516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.412189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.412533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.413896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.414933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455526-3e8a-423b-a258-455df18ec7fb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.416167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.424651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 493/981] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:13.424651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.452794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.453441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.453721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.454030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.454673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.456066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:13.468486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:13.471067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:13.472096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 489/982] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:13.473055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.476553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.542556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641408-dc8a-4eea-bb51-e9c4316959d9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.579249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.579875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.580151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.580419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.581224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.581521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.583385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.589604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc0e-f900-4f49-ad01-768f6594049e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.602776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 494/983] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:13.603635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.631374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.631913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.632193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.632463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.633142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.633449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.634857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.639757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 490/984] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:13.640552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.678557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.708750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.733989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94a966-2207-4fd0-b6ff-7019734298bf None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.752792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae592f8-8c48-42eb-80fb-a48cc4a7638f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.774804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.775450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.775736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.776015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.776709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.777020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.778441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.784277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 495/985] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:13.785116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.791204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.792059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.792444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.792831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.868315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.869797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.917685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330c60-8804-4af6-b45e-1df2ad924faa None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:13.948486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.953017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:13.968516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:13.969157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:13.969430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:13.969706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:13.970536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] RBAC: Adding query filter params (interface=public, service_id=4eaa5a22a6e5412bafa6c9ba5a6d3bd7, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:13.970844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:13.972227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:13.982585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 496/986] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/endpoints?service_id=4eaa5a22a6e5412bafa6c9ba5a6d3bd7&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:13.982585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:13.990047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 491/987] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:13.990913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.087952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7a8b9-cb9c-492f-be85-f43f5b07cd14 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.111942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6436b4-4798-480e-87db-dcb1d9f7f10d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.138817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.139969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.140241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.140510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.141206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:14.141514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.142921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.147095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.147726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.148005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.148278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.148963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.155391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 497/988] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:14.155602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.156012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.167277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 492/989] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:13 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:14.167277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.198642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:14.220826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:14.289877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8235bbc-8f95-4035-bb56-7d352ef4a45f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.300795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaaf95-a5e2-41a6-9f0e-c05417730ab8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.335530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.336485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.336773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.337045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.351174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.351931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.352121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.352907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.352907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.353394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.354077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] RBAC: Adding query filter params (interface=admin, service_id=ed58f434f888496d84145cf327c79f42, region_id=RegionOne) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:14.354469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.356612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.372589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 493/990] 10.222.0.48 () {56 vars in 1249 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/endpoints?service_id=ed58f434f888496d84145cf327c79f42&interface=admin®ion_id=RegionOne => generated 193 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:14.378176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:14.379422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:14.380559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:14.381972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:14.381972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:14.381972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:14.381972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:14.381972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 498/991] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:14.390306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.492446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:14.502525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b887ac-7ca1-423b-8db2-eb89355868cc None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.504737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074035-842e-4274-9a5b-e9bd5c9db459 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.506595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:14.541080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.541807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.542122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.542399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.543063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.544470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.544826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.545743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.545889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.546135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.547098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:14.547216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.548820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.553062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 494/992] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 08:59:14.570148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 499/993] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:14.579551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.751685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423e02b-21e2-4a4c-a8a3-c5c72a261800 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:14.789237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:14.789982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:14.790269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:14.790546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:14.805989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:14.808581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:14.828693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:14.829805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:14.830743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:14.833913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 495/994] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:14.836512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:14.971404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32044d-bcb5-4652-9897-09b45cd2f380 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:15.019198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:15.020141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.020617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.021079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.022442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:15.023055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:15.024622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e66f96-bc6f-41df-9383-3e44b964b6ed None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:15.043080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 500/995] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:15.044393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:15.066786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:15.158699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e388e94-716d-4b32-81f4-fb3f280df9f8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:15.196112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:15.196745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:15.197028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:15.197305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:15.198136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] RBAC: Adding query filter params (role.id=83477ffa00a4409bb4929e9c4b383c8f, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=e0d7c7e659ec440786e599f1c0a64d48) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:15.198446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:15.200690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ce820-510f-4d33-85f7-289356c675ab None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:15.209887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 496/996] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:15 2022] GET /identity/v3/role_assignments?user.id=e0d7c7e659ec440786e599f1c0a64d48&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=83477ffa00a4409bb4929e9c4b383c8f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:16.445299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522ca5-e535-468c-ba39-62cfbede44ff None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:16.446126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522ca5-e535-468c-ba39-62cfbede44ff None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:16.446264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522ca5-e535-468c-ba39-62cfbede44ff None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:16.447851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 501/997] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:16.453888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:16.454380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:16.454461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:16.473578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:16.475268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:16.580914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 497/998] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:16.606284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522ca5-e535-468c-ba39-62cfbede44ff None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:16.631781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:16.735888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522ca5-e535-468c-ba39-62cfbede44ff None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:16.775589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:16.776265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:16.776562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:16.776836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:16.777430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:16.778848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:16.788320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:16.789386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:16.794785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 502/999] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:16.798387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:16.815486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:16.915409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e887aef-839b-405f-8f4f-e0f2c9ebf7af None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:16.961725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:16.963079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:16.963753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:16.964419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:16.965778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:16.966544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:16.969662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:16.980520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 498/1000] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:16.981800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.093857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.111925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdba5-906f-4d26-a92e-422320a1df2f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.158261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.159296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.159778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.160268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.161155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:17.161657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.163402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f88550-a0bb-46b0-aaaa-12eaebf4f2d4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.174094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 503/1001] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Jul 28 08:59:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:17.175408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.279375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.299711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b04-44c6-47c2-95b9-e32dafafc8d5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.300393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b04-44c6-47c2-95b9-e32dafafc8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.300674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b04-44c6-47c2-95b9-e32dafafc8d5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.302108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 504/1002] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:17.303171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bcc19-8d75-4e71-b017-8606cdc78a79 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.303481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bcc19-8d75-4e71-b017-8606cdc78a79 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.303757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bcc19-8d75-4e71-b017-8606cdc78a79 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.304987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 505/1003] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:17.308888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.309233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.309514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.309628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3f88d-8602-431f-9e67-8d876a1b2224 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.328137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:17.329344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:17.348707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.349563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.349864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.350164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.350921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.352460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12e82f-06c2-48e1-bf4a-bcfa0fc58a06 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.385290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 499/1004] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:17.386826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.387119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.387400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.408855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:17.410149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:17.419354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 506/1005] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:17 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:17.445047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.505895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 500/1006] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:17 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:17.528250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.557942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.586037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4a19f-8022-4dbb-8838-761010d9a707 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.602380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.647732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.648216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.648493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.648761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.649395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.651301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:17.664053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:17.665119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:17.666015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 507/1007] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:17.667967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c02c8-154f-43dd-903c-c37daffd3f6f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.671645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.713684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.714375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.714659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.714939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] PATH_INFO: `/v3/services/image` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.715585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.716980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:17.729215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:17.730457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:17.731273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 501/1008] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:17.739243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.800319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65658041-aaa9-419c-a881-ad2f86c706fc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.859336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.860321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.860812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.861310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.862637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:17.863012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.864725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.871618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 508/1009] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:17.879089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:17.883741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.905491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fc5e7-5437-497f-9891-7b4764b19c3e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:17.945105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:17.967929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:17.969444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:17.969727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:17.970037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:17.970742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:17.971052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:17.972460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:17.982366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 502/1010] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:17.991176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:18.056792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6184-44b7-4e27-98ad-2bf5413c8d49 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:18.112742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:18.113649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.115796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.115796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.115796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:18.115796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:18.117181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:18.122826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 509/1011] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:18.129711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:18.147639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf9c1-ef39-4b5f-a1e1-5e1a062380a4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:18.201354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:18.202554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.202834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.203111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.203801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:18.204108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:18.205572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31a17b-e3b1-457e-a3ae-89895278f7a1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:18.221937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 503/1012] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:18.254172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:18.303123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031eed6-b3cd-40ae-84ea-2bafae3562c8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:18.349020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:18.349661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:18.349986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:18.350240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:18.351032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:18.352429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf06454-7d3c-4f67-a7cc-7dfcf35d5416 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:18.375492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 510/1013] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:19.730126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9cc11-4cc5-48c5-9fb1-8fd4ad0fd49d None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.730701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9cc11-4cc5-48c5-9fb1-8fd4ad0fd49d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.730969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9cc11-4cc5-48c5-9fb1-8fd4ad0fd49d None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.733656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 504/1014] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:19.739514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:19.740260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:19.740399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:19.760118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:19.761309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:19.856950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 511/1015] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:19 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:19.881152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9cc11-4cc5-48c5-9fb1-8fd4ad0fd49d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:19.960320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.029379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9cc11-4cc5-48c5-9fb1-8fd4ad0fd49d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.081633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.082322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.082607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.082881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.097752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:20.107177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:20.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:20.109532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:20.110738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 505/1016] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:20.115668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.195357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.255893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eef85-c882-41df-9d43-d6fdcb678dd2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.292892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.293538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.293817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.294117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.294851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:20.295156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.297144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda2f8e-e8bc-4ca4-94fc-6e35d4f6bd86 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.306861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 512/1017] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:20.313463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.423291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580208a-c555-4452-96c8-05b83de8ac1c None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.423691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580208a-c555-4452-96c8-05b83de8ac1c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.424042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580208a-c555-4452-96c8-05b83de8ac1c None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.425772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 513/1018] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:20.430994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.431302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.431807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.446196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1b1ad-f540-488e-bc00-ae82928eeb1b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.456029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:20.457892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:20.461397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.486607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.487715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.488207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.488698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.506914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.508403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:20.524749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:20.525863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:20.526830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:20.526830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:20.526830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:20.526830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:20.530444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 506/1019] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:20.532149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.569964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 514/1020] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:20 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:20.590144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.643879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.680697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567f976-d626-4e93-876f-9237c4b9e091 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.725413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.726078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.726361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3579b-3a21-4c66-8ecc-030199f6d77d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.726501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.726632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.727322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:20.727626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.729011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a125a4-8c07-46a9-95b1-b3e9c8ad241f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.741527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 507/1021] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:20.742951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5fbf-fe3f-4967-87cc-17c9f4cbc31b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.743236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5fbf-fe3f-4967-87cc-17c9f4cbc31b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.743516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5fbf-fe3f-4967-87cc-17c9f4cbc31b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.744776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 508/1022] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:20.747917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d5fbf-fe3f-4967-87cc-17c9f4cbc31b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.773163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.774776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.775568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.776029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.777221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.779497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:20.779835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874a628-f19f-4989-9c3a-b8c9a8b1082d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:20.813874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 515/1023] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:20 2022] POST /identity/v3/services => generated 242 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:20.813874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.813874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.813874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.826261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:20.828072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:20.909899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d5fbf-fe3f-4967-87cc-17c9f4cbc31b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:20.924850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 516/1024] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:20.961275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:20.977120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:20.977747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:20.978051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:20.978330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:20.987563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:20.989229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:21.005907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:21.007499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:21.008752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:21.011600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 509/1025] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:21.015869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.018824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.095910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9692b3a-3722-49ac-af97-fd40e5c008c1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.130392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.139662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.140868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.141328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.141821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.143877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.145358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.147064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 517/1026] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:21.153351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.162700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee814-bea4-4869-a640-eec37e980ec6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.211208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.211841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.212121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.212395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.213192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:21.213499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.214963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.250819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 510/1027] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:21.252130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.288159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd8da-57c7-4296-9671-9af40a6b7e3d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.292926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.343835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.344472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.344750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.345029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.345719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.347313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.361220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b214093-7ec2-4e1f-84a9-c76ce8a94d67 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.385918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 518/1028] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:21 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:59:21.421766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.422854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.423317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.423764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.438902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:21.507601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.509224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.658649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 511/1029] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:21.664574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.779209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8264dd0-2708-4da4-b85d-8fc045b79697 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:21.823455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:21.824114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:21.824395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:21.824667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:21.836755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:21.838247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:21.854587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:21.855772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:21.856660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:21.856660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:21.856660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:21.856660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:21.857919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 519/1030] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:21.861647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:21.926727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:22.004994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720d301-e941-4dc9-8085-9916bebd95b7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:22.051794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:22.052674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.053138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.053589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.054487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:22.054939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:22.056537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:22.071252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 512/1031] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:22.079749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:22.092187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:22.194457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6dac-a50f-461c-bccf-2943edd3b3de None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:22.232489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:22.233471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.233924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.234345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.241720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:22.243392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:22.255771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:22.256781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:22.257679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 520/1032] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:22.263687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:22.385314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:22.389918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669dbd-a102-468f-b4c2-2de22bbde865 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:22.426206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:22.427136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.427624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.428079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.429016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:22.429506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:22.431157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1661b9b-6b6c-4099-bd33-ff3c46775d8f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:22.445736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 513/1033] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:22.451599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:22.534806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:22.573406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37301361-90ec-4ba0-819f-c41a7847e127 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:22.611881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:22.612541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.612832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.613103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.613945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] RBAC: Adding query filter params (role.id=83477ffa00a4409bb4929e9c4b383c8f, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:22.614266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:22.615726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda10b36-41b4-4298-ae9a-0eb4d1137854 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:22.622202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 521/1034] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:22 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=83477ffa00a4409bb4929e9c4b383c8f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:22.894500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed58c7-1635-4723-afff-55c9412ba26d None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.895318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed58c7-1635-4723-afff-55c9412ba26d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.895318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed58c7-1635-4723-afff-55c9412ba26d None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.896724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 514/1035] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:22.902229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:22.902710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:22.902795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:22.924107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:22.926453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:23.014094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 522/1036] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:23.039381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed58c7-1635-4723-afff-55c9412ba26d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.156239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed58c7-1635-4723-afff-55c9412ba26d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.197633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.198342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.198624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.198948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] PATH_INFO: `/v3/services/image` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.199594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.200976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:23.210933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:23.211961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:23.212882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:23.214662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 515/1037] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:23.218473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.292407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:23.355361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de26d6-0fe8-46e0-b32b-e87ad9b94dc6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.404383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.405026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.405307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.405704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.406554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] RBAC: Adding query filter params (name=image) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:23.406809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.408184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb840b1-f237-4dfe-951a-47d192416299 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.418498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 523/1038] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:23.420662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.459709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:23.528907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1b2f1-b68d-4410-bef7-4567497d7b88 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.529260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1b2f1-b68d-4410-bef7-4567497d7b88 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.529458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1b2f1-b68d-4410-bef7-4567497d7b88 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.531690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 524/1039] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:23.536638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.537000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.537261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.552627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef775-7af6-43df-af14-90dc74ba7926 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.566772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:23.569431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:23.591490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.592998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.593505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.593777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.594512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] RBAC: Adding query filter params (type=image) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:23.594818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.596207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.605892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 516/1040] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:23.609261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.668245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 525/1041] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:23.692759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.723918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:23.740500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6417e-ee55-4342-aef1-675eebe632de None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.777388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:23.787335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.787335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.787335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.787335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.789560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] RBAC: Adding query filter params (interface=public, service_id=1ffb36721c434f4e87b151d5d581e4ba, region_id=RegionOne) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:23.789560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.789560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.795000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 517/1042] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/endpoints?service_id=1ffb36721c434f4e87b151d5d581e4ba&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:23.801896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.833763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385b7d-d2b6-4a55-a7f2-578582c554af None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.876185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.876865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.877145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.877414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.878918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.880379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.886413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 526/1043] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:23.890136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:23.915874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370db9-cc93-4011-a928-98e582adb4ce None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:23.966111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:23.966918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:23.967265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:23.967622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:23.969434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:23.971156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cb336-9104-488c-9dde-e9f66b0e89e7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:23.984190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 518/1044] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 08:59:23 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 08:59:24.023907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f769-0c2c-4ce0-a339-43410ed4cfee None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:24.057237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:24.072016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:24.072976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.073410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.073832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.074738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:24.076963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b33c-1af2-4a00-a0c2-b7b4a3d747a2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:24.109912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 527/1045] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:59:23 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 08:59:24.661023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329b52c-ceec-4d69-9c75-fc87c3d4dd7a None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.661464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329b52c-ceec-4d69-9c75-fc87c3d4dd7a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.661596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329b52c-ceec-4d69-9c75-fc87c3d4dd7a None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.663138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 519/1046] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:24.668509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.668810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.669071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.689370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:24.690603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:24.774193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 528/1047] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:24.796346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329b52c-ceec-4d69-9c75-fc87c3d4dd7a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:24.818111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:24.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329b52c-ceec-4d69-9c75-fc87c3d4dd7a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:24.968404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:24.969051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:24.969391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:24.969672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:24.980758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:24.986664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:24.993231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:24.994348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 520/1048] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:24.996390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.109642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4061bd-330e-40b5-baba-93fd2a984b41 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.121400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.149280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.149972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.150255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.150531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.151272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:25.151577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.152988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.169884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 529/1049] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:25.172764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.278729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.309027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bad93-605c-4e60-88b6-0e3a355573ca None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.349861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.350540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.350809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.351082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.362820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.364282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:25.379661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:25.380619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:25.386419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:25.386419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:25.386419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:25.386419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:25.386419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 521/1050] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:25.388361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.499735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016143-74b7-4a85-9002-32e61c932b5d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.535367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.535940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.536201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.536463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.537163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:25.537465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.538866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.556259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 530/1051] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:25.558126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.672530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac233-d7cf-4497-9728-7d065387cb4b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.700787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.710596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.711251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.711534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.711802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.721976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.723439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:25.735308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:25.736522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:25.737460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:25.741882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 522/1052] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:25.742853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:25.782973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:25.864310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783c8e-c57f-4b32-986e-dfeefb10fb75 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:25.901536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:25.902189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:25.902468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:25.902738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:25.903546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:25.903847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:25.905303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7304e-17eb-4d03-a97a-364558c30c69 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:25.936770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 531/1053] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:25.942805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.067659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03deb6b-dcbd-4be7-8138-4ba80fcf6e29 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.085037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d189c-c0f5-4521-a4d7-4964484f1090 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.085429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d189c-c0f5-4521-a4d7-4964484f1090 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.085690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d189c-c0f5-4521-a4d7-4964484f1090 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.089868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 532/1054] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:26.093323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041545-a7e5-4b11-bdfb-509219694f7b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.093612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041545-a7e5-4b11-bdfb-509219694f7b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.093924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041545-a7e5-4b11-bdfb-509219694f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.105517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.106498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.106961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.107504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.114154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d7041545-a7e5-4b11-bdfb-509219694f7b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:26.115946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d7041545-a7e5-4b11-bdfb-509219694f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:26.119316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.120780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:26.137785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.138791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.140483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:26.140483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:26.140483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:26.140483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:26.140483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 523/1055] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:26.145366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.194796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 533/1056] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:26.203711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.213551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e3462-781a-4c73-9710-59e9b939bd65 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.213902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e3462-781a-4c73-9710-59e9b939bd65 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.214359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e3462-781a-4c73-9710-59e9b939bd65 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.215641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 534/1057] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 08:59:26.217997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e3462-781a-4c73-9710-59e9b939bd65 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.241526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.266256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7a98-304a-4808-be84-0d4fb61826ab None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.303423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.304091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.304427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.304708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.305520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.305909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.307802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ec700-19cd-48f2-a470-b63df7410da2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.322145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 524/1058] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:26.323501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.323791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.324069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.331942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e3462-781a-4c73-9710-59e9b939bd65 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.342429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:26.343659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:26.370546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.371104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.371377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.371643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] PATH_INFO: `/v3/services/image` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.372271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.374217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.383304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.384491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:26.385325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 535/1059] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:26.386165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.414695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 525/1060] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:26.415380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.448775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.479018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.496036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762bcdd-20ec-43ad-bd15-4c6e03a27bef None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.521617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae6d0-720d-485a-8583-eb4b653a2d05 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.530685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.531334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.531615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.531894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.538679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.540126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:26.552696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.553906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 536/1061] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:26.559972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.566625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.567257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.567642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.567916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.568594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] RBAC: Adding query filter params (name=image) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.568894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.570353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.575449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 526/1062] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.576224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.658520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0148564-2e09-46dd-92ef-c809683278c4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.667983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.675624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0adb-710e-4503-b531-f2be8692f17d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.692766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.693267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.693551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.693828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.704314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:26.709801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:26.711027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 537/1063] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:26.711956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.715521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.720364 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.721010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.721295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.721792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.722372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.722676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.724122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.738250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 527/1064] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.739159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.828845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adba44-5696-4c8b-bcd1-8ead818e6f22 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.860059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ff7df-2057-4c42-ab10-35918aaf88d4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:26.872149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.872774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.873049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.873479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.874189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] RBAC: Adding query filter params (type=image) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.874502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.875867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.881220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 538/1065] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.881948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.896378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:26.897001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:26.897277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:26.897546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:26.898289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:26.898589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:26.900184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:26.911054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 528/1066] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:26.911806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:26.934228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.952622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:26.988703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27641a92-e120-47cf-834b-628bcf65ac85 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.023054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.023754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.024018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b9135-a1a2-493c-8ef1-2a030bc6303c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.024459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.024459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/83477ffa00a4409bb4929e9c4b383c8f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.026528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=96a7013052fd4d48a2463cf51f13861f, role_id=83477ffa00a4409bb4929e9c4b383c8f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.027961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.057657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 539/1067] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:26 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/96a7013052fd4d48a2463cf51f13861f/roles/83477ffa00a4409bb4929e9c4b383c8f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:27.057657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:27.067162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.067814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.068100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.068374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.069109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.070605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.105876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 529/1068] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 08:59:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:27.180772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069103e-2fe3-4a25-b5cc-6583d2e6cae8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.216132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:27.244437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.245114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.245391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.245670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] PATH_INFO: `/v3/users/936c5205e07f48548ffe6735a0ddf64a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.261803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] RBAC: Authorizing `identity:get_user(user_id=936c5205e07f48548ffe6735a0ddf64a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.263303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.269873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 540/1069] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:27 2022] GET /identity/v3/users/936c5205e07f48548ffe6735a0ddf64a => generated 285 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:27.271711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:27.353506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:27.388817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9187-cdaa-4069-860d-644156b19a2f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.423794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.424426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.424706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.424980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.496795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.498205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.578829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:27.617743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 530/1070] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:27.624326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:27.736201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569468f7-896a-431c-bb19-0186172463f9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.773372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.774275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.774720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.775156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.777095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.779580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c8424-4872-4b71-aa3f-e069fa98735b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.782365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 541/1071] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:27 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:27.788286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:27.909188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18bf9-6ea2-4218-93b3-24626ff615be None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:27.954981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace, user.id=936c5205e07f48548ffe6735a0ddf64a) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:27.963694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24687036-94b4-4db6-a6c8-5d146427776e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:27.975112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 531/1072] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:27 2022] GET /identity/v3/role_assignments?user.id=936c5205e07f48548ffe6735a0ddf64a&scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:29.108985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88330c-5d5b-4212-b235-d34295e0a924 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.109527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88330c-5d5b-4212-b235-d34295e0a924 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.109527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88330c-5d5b-4212-b235-d34295e0a924 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.111048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 542/1073] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:29.116913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.117455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.117455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.137303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:29.138548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:29.169717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.176227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45143-dd6b-4d15-9ce9-4f6e26eceecf None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.176450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45143-dd6b-4d15-9ce9-4f6e26eceecf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.176739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45143-dd6b-4d15-9ce9-4f6e26eceecf None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.178759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 543/1074] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:29.184530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.185061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.185408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.205155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:29.206408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:29.226398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 532/1075] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:29.251263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.268089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.291184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 544/1076] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:29.314624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.368315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a86b56-59ef-411c-b1bf-b12b67bd4862 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.406371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.407387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.407678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.408213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.421162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:29.426217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:29.427258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:29.428223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 533/1077] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:29.433965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.445470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d9869-e6ec-4331-92b5-551fec3a1949 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.465001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.501170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.501810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.502118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.502394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.559588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.575632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193bc27-d20a-4cc5-baad-d763c945da6c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.582812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:29.584428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:29.621696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.622491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.622779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.623057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.623875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:29.624283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:29.625883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:29.635880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 534/1078] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:29.642045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.722503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 545/1079] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:59:29.723694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.751202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.769000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63bbe-a798-43fc-8124-4f1ed7a903ee None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.806192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.807178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.807495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.807770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.815179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:29.821563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:29.823288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:29.840972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2835-894b-45a8-825e-262762998bc5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:29.841463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:29.843309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:29.844273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:29.844273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:29.844273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:29.844273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:29.844273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 535/1080] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:29.849323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.886422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:29.887071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:29.887350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:29.887624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:29.902543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:29.905222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:29.930224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:29.931188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:29.932334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:29.932334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:29.932334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:29.932334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:29.932554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 546/1081] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:29.938366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:29.970417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17def9ac-2468-4fea-9941-4a41c68e0501 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.008659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.009202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.009482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.009759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.010502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.010805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.012178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.022995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.031624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 536/1082] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.032956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.058290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3e36-1d00-409e-81c8-79fa4158df4c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.089457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.095980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.096614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.096902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.097165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.097884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.098194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.099592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a9239-60aa-4961-936b-d990bd8e3386 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.111996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 547/1083] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.113263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5756-307b-4cf4-8068-ec396def7137 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.113550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5756-307b-4cf4-8068-ec396def7137 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.113827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5756-307b-4cf4-8068-ec396def7137 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.115128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 548/1084] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:30.120925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.121210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.121485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.141865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:30.143118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:30.149885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab97fb-a30b-4912-9e12-09935b3d8c0b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.191409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.192130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.192431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.192693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.201120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.203507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 08:59:30.217563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:30.218838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:30.219782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 537/1085] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 08:59:30.224865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.232075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 549/1086] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:30.262383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.302005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.356881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680864-7313-4199-a88d-07a387ca78cc None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.389775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0b2ec-d9d8-45f7-adc9-8e5da606c64d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.404982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.406319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.406669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.407098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.408184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.408496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.410169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.416819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.428329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 538/1087] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.431389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.432306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.432580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.432852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.434327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.444674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:30.449588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:30.450759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:30.452540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 550/1088] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:30.457239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.544978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f22c-47ed-4c67-9740-4df4498c3b95 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.565549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.571822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279cd24-4fba-4b37-bda7-1a5c19dc6279 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.586515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.587182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.587510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.587782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.611451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.612076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.612402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.612643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.613376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:30.613679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.615096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.624526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 551/1089] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.631978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.663091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.664573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.678740 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.734163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494440-af2b-4c91-9557-65d481c1b33e None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.782721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.783764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.789063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 539/1090] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:59:30.789283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.789283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] PATH_INFO: `/v3/users/936c5205e07f48548ffe6735a0ddf64a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.789283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=936c5205e07f48548ffe6735a0ddf64a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.789283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.790610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.791104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 552/1091] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/users/936c5205e07f48548ffe6735a0ddf64a => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.799734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.813053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.913677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2107c-0a59-478f-afa1-30408e4b5f04 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.934064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89181835-34f9-4d1a-b7bc-fb6de6a68ab4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:30.950627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.951285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.951602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.951782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:30.951935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.963608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.965093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.976843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:30.977598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:30.977873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:30.978167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:30.979349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:30.980795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 08:59:30.980928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:30.981831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:30.983769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 08:59:30.983769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 08:59:30.983769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 08:59:30.983769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:30.983769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 540/1092] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:30.984206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 553/1093] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:30.988832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:30.989271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.066962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.104036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d8c18-633c-4702-8a8e-95993145651c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.104691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1093-2342-4ad6-b365-4eff03e7957f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.138249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.138886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.139166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.139443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace/users/936c5205e07f48548ffe6735a0ddf64a/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.140973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.141472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] RBAC: Authorizing `identity:create_grant(project_id=87a15a6cd7374a1b8a23ec2f6a464ace, user_id=936c5205e07f48548ffe6735a0ddf64a, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.141603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.141915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.142194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.142822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.142925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:31.143192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.144555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.157816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 554/1094] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 08:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:31.161566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 541/1095] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:30 2022] PUT /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace/users/936c5205e07f48548ffe6735a0ddf64a/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:31.167145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.214956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.294550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8c8a-11eb-43a8-8347-c38d976485aa None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.334410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.334986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.335265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.335538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.347059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.348520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 08:59:31.366098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:31.371170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:31.372062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 555/1096] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:31.373572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.480561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769832cc-77ee-4000-aa97-3b49118b924a None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.509215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.516183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.516878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.517229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.517524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.518496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:31.518978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.520608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91268fd-bf73-407c-b4e3-08b99b8291ba None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.533379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 542/1097] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:31.539369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:31.660779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:31.663865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c6720-74ec-4fee-9898-4d1a0288d18f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:31.698752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:31.699380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:31.699663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:31.699938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:31.700741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] RBAC: Adding query filter params (role.id=83477ffa00a4409bb4929e9c4b383c8f, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=96a7013052fd4d48a2463cf51f13861f) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:31.701057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:31.703174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e884cf-d237-4d94-ab6f-a095de970a33 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:31.709547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 556/1098] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:31 2022] GET /identity/v3/role_assignments?user.id=96a7013052fd4d48a2463cf51f13861f&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=83477ffa00a4409bb4929e9c4b383c8f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:33.202551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17d9e-74d5-4e34-9dea-ef1a3e1ca44c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.203069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17d9e-74d5-4e34-9dea-ef1a3e1ca44c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.203069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17d9e-74d5-4e34-9dea-ef1a3e1ca44c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.204360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 543/1099] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:33.209671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.210195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.210304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.228997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:33.230239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:33.297324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 557/1100] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:33.321047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea17d9e-74d5-4e34-9dea-ef1a3e1ca44c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:33.425469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea17d9e-74d5-4e34-9dea-ef1a3e1ca44c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:33.459929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:33.460495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.460775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.461049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.472083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:33.477061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:33.478279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:33.479421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 544/1101] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:33.485067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:33.578020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:33.585117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971a5b1-d326-43dd-a098-84ea2691114b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:33.619812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:33.620434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.620715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.620989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.621709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:33.622065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:33.623442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:33.631734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 558/1102] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:33.637612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:33.697420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:33.749206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4486c8f-1330-40f7-81f7-0c3c0e3f616c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:33.786539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:33.787209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.787486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.787761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] PATH_INFO: `/v3/users/936c5205e07f48548ffe6735a0ddf64a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.788984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] RBAC: Authorizing `identity:get_user(user_id=936c5205e07f48548ffe6735a0ddf64a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:33.790470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:33.792106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 545/1103] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:33 2022] GET /identity/v3/users/936c5205e07f48548ffe6735a0ddf64a => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:33.797952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:33.906368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95834f4-c213-4f02-aebb-abab5d1e2212 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:33.941894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:33.942524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:33.942801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:33.943068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:33.969336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:34.017187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:34.019108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:34.121007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 559/1104] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:34.126971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:34.226290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa7e4-1e01-496b-a798-60b21e7a6efb None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:34.244587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:34.261589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:34.262267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:34.262547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:34.262819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:34.263957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:34.265415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bad0da-0a7d-4a53-aa98-f686015aea26 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:34.267118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 546/1105] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:34 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:34.273381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:34.343890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:34.386959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d00a-5c2f-42d4-ae22-ac0cbffc7f52 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:34.421738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:34.422386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:34.422664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:34.422935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:34.423722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace, user.id=936c5205e07f48548ffe6735a0ddf64a) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:34.424025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:34.425449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278491c-e74b-48aa-bdd6-6a2802c23d3c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:34.431619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 560/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:34 2022] GET /identity/v3/role_assignments?user.id=936c5205e07f48548ffe6735a0ddf64a&scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:36.524439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fd4-447d-4a63-b03a-6988acd19d86 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.525042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fd4-447d-4a63-b03a-6988acd19d86 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.525042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fd4-447d-4a63-b03a-6988acd19d86 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.526392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 547/1107] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:36.532511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.533032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.533032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.552835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:36.554477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:36.623809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 561/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:36.648435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0fd4-447d-4a63-b03a-6988acd19d86 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:36.759039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0fd4-447d-4a63-b03a-6988acd19d86 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:36.795058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:36.795708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.795991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.796267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.797444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:36.798910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370c0e6-a544-4e71-91d0-83e3af2a1caa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:36.800528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 548/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:36.806426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:36.856191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:36.907741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7dbc5-082c-455d-a420-d607a609e880 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:36.945452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:36.946116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:36.948117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:36.948117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:36.948117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:36.949371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde06f4-7539-4d9c-ba14-901d7b579c7d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:36.979501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 562/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:59:36 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:39.042531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183dc76-dc56-434e-8214-65feaf7c926b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.043092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183dc76-dc56-434e-8214-65feaf7c926b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.043092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183dc76-dc56-434e-8214-65feaf7c926b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.044421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 549/1111] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:39.050412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.050854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.051052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.077710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:39.078929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:39.144877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 563/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:39.168049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183dc76-dc56-434e-8214-65feaf7c926b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:39.226291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:39.272744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183dc76-dc56-434e-8214-65feaf7c926b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:39.308719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:39.309368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.309652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.309960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] PATH_INFO: `/v3/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.311456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] RBAC: Authorizing `identity:get_role(role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:39.312844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:39.314251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 550/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:39 2022] GET /identity/v3/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:39.320189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:39.354506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:39.430182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe982f93-2e7f-430b-aef6-d0dca839fb4f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:39.463564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:39.464215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.464499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.464776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] PATH_INFO: `/v3/users/9a0cbfea690a48698df35ebbd2875778` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.478583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] RBAC: Authorizing `identity:get_user(user_id=9a0cbfea690a48698df35ebbd2875778)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:39.479996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:39.481764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 564/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:39 2022] GET /identity/v3/users/9a0cbfea690a48698df35ebbd2875778 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:39.488275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:39.591113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5672dc5-e070-4985-82ad-94a3547d239b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:39.624733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:39.636449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:39.637074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.637356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.637629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.706369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:39.707834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:39.815798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 551/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 08:59:39.821725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:39.925475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:39.932824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2aaee-4d30-4d95-a9d4-0864f0a6ff69 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:39.972102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:39.972775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:39.973091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:39.973429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:39.974837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:39.976350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ec4f-c179-46bf-a206-e4a226ff1aa6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:39.978159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 565/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:39 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:39.984413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:40.009327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:40.088619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0502dc-456f-4831-8ce7-649f04a014fa None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:40.122326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:40.122945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:40.123760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:40.124032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:40.124814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace, user.id=9a0cbfea690a48698df35ebbd2875778) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:40.125119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:40.126580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59c5fc-508c-4796-aacd-0bbd0be009b9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:40.132124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 552/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:39 2022] GET /identity/v3/role_assignments?user.id=9a0cbfea690a48698df35ebbd2875778&scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:42.126124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c577-13da-48a2-ad74-c9c03f77f0e4 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.126676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c577-13da-48a2-ad74-c9c03f77f0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.126776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c577-13da-48a2-ad74-c9c03f77f0e4 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.128155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 566/1118] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:42.133476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.133966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.134084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.152031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:42.153247 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:42.226226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 553/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:42.249530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2c577-13da-48a2-ad74-c9c03f77f0e4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.373707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2c577-13da-48a2-ad74-c9c03f77f0e4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.426164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.426823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.427106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.427384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] PATH_INFO: `/v3/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.428867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] RBAC: Authorizing `identity:get_role(role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.430276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.431663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 567/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:42 2022] GET /identity/v3/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:42.438188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.455482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.538403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc28698-96c0-472c-bb1e-0145b53ee503 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.575992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.577194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.577471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.577743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] PATH_INFO: `/v3/users/9a0cbfea690a48698df35ebbd2875778` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.578938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] RBAC: Authorizing `identity:get_user(user_id=9a0cbfea690a48698df35ebbd2875778)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.580374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.581998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 554/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:42 2022] GET /identity/v3/users/9a0cbfea690a48698df35ebbd2875778 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:42.587916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.618636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.691898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2cc23-e536-4eda-819f-aa8a9c159259 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.725779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.726462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.726742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.727014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.728178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.729611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6807d-67f1-4e1f-8abe-3c8f4a549c8d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.731337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 568/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:42 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:42.738010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:42.823568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:42.840254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc305938-dbdc-4087-b914-7bc24cbdc214 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:42.876562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:42.877238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:42.877520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:42.877797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace/users/9a0cbfea690a48698df35ebbd2875778/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:42.880026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] RBAC: Authorizing `identity:create_grant(project_id=87a15a6cd7374a1b8a23ec2f6a464ace, user_id=9a0cbfea690a48698df35ebbd2875778, role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:42.881472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a30b26-47eb-4a9b-a026-0badabc171ad None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:42.895601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 555/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:42 2022] PUT /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace/users/9a0cbfea690a48698df35ebbd2875778/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:44.897288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318e14-1896-431a-a8f1-6adfbc77a4f2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:44.897957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318e14-1896-431a-a8f1-6adfbc77a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:44.897957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75318e14-1896-431a-a8f1-6adfbc77a4f2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:44.899241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 569/1124] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:44.905450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:44.906005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:44.906005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:44.924440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:44.925647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:44.993611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 556/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 08:59:45.017563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75318e14-1896-431a-a8f1-6adfbc77a4f2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.127407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.132785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75318e14-1896-431a-a8f1-6adfbc77a4f2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.168212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.168765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.169051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.169325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] PATH_INFO: `/v3/roles/9421cdb84ee5428b9542a3cbba296f8f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.170898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] RBAC: Authorizing `identity:get_role(role_id=9421cdb84ee5428b9542a3cbba296f8f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.172275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.173658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 570/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/roles/9421cdb84ee5428b9542a3cbba296f8f => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.179675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.253772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.281737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0794ec5-37f8-4229-8481-524c5b8a0530 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.317339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.318036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.318318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.318593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] PATH_INFO: `/v3/users/9a0cbfea690a48698df35ebbd2875778` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.319776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] RBAC: Authorizing `identity:get_user(user_id=9a0cbfea690a48698df35ebbd2875778)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.321210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.322886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 557/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/users/9a0cbfea690a48698df35ebbd2875778 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.329526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.433222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6339b-8523-4cbb-9bc9-800d65dc9ea4 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.468406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.469028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.469309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.469583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.503335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.543406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.544896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.644738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 571/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:45.650808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.757118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d036b1-3715-481b-8cff-9caae21e0efa None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.791168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.791815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.792096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.792372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] PATH_INFO: `/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.793559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] RBAC: Authorizing `identity:get_project(project_id=87a15a6cd7374a1b8a23ec2f6a464ace)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.795531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6b3b-29e9-4482-bdf0-0b8575b037ae None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.797219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 558/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/projects/87a15a6cd7374a1b8a23ec2f6a464ace => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:45.803517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:45.877210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:45.913140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6e71-10fa-4245-8247-767fe551f705 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:45.952850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:45.953470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:45.953747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:45.954046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:45.954836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] RBAC: Adding query filter params (role.id=9421cdb84ee5428b9542a3cbba296f8f, scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace, user.id=9a0cbfea690a48698df35ebbd2875778) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:45.955141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:45.956566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c851c29-e9df-4a86-8e90-c2717e3fa2e2 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:45.962541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 572/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:45 2022] GET /identity/v3/role_assignments?user.id=9a0cbfea690a48698df35ebbd2875778&scope.project.id=87a15a6cd7374a1b8a23ec2f6a464ace&role.id=9421cdb84ee5428b9542a3cbba296f8f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:47.990186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d975-e507-461a-aebc-51359e4e1eef None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:47.990726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d975-e507-461a-aebc-51359e4e1eef None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:47.990726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116d975-e507-461a-aebc-51359e4e1eef None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:47.992082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 559/1131] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:47.997430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:47.997926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:47.998050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:48.017791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:48.019013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:48.086210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 573/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:48.109803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116d975-e507-461a-aebc-51359e4e1eef None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:48.122354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:48.227131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116d975-e507-461a-aebc-51359e4e1eef None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:48.261739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:48.262422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:48.262704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:48.262982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:48.264731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:48.266208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eb9a5-d38d-4db5-bfa6-e032d56ebb47 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:48.267832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 560/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:48.274041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:48.339096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:48.379974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493339e-0ac9-4680-b838-fe96bf799e54 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:48.413898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:48.414521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:48.414798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:48.415066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:48.415747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:48.417177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df723a-2096-4875-baeb-0f962b663dd3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:48.445483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 574/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 08:59:48 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:50.501010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389949-f116-459a-bd24-e706ed0c3e77 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:50.501554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389949-f116-459a-bd24-e706ed0c3e77 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:50.501554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389949-f116-459a-bd24-e706ed0c3e77 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:50.502988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 561/1135] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:50.508440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:50.508863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:50.509004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:50.527257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:50.528462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:50.597458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 575/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:50.622366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389949-f116-459a-bd24-e706ed0c3e77 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:50.703172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:50.726935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389949-f116-459a-bd24-e706ed0c3e77 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:50.761581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:50.762168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:50.762448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:50.762720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:50.763929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:50.765361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d43762-5519-45fc-b828-caa5559f7ff1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:50.767013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 562/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 08:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:50.772848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:50.817920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:50.876886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d031c6-2c93-41d6-a742-f189199d111a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:50.911743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:50.912364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:50.912642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:50.912908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:50.913586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:50.915062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75383f3-8199-432c-9d6a-ca23060efd8b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:50.938546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 576/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 08:59:50 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 08:59:53.023968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99c53-119d-47b0-8873-7bd4dcc53288 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.024494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99c53-119d-47b0-8873-7bd4dcc53288 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.024619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99c53-119d-47b0-8873-7bd4dcc53288 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.025917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 563/1139] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:53.031777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.032233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.032326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.054972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:53.056179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:53.134146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 577/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:53.159042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99c53-119d-47b0-8873-7bd4dcc53288 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:53.258540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99c53-119d-47b0-8873-7bd4dcc53288 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:53.293149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:53.293782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.294087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.294359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.304806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:53.310029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:53.311196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:53.312488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 564/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:53.317316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:53.345905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:53.421595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677de6bd-b90c-4fd0-b6cd-4e96c069ed26 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:53.455348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:53.455966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.456242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.456511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.457222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:53.457522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:53.458924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:53.468071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 578/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:53.473955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:53.483565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:53.582166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6ccca-9567-4822-b07b-5fdc5bfdbce4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:53.615886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:53.616527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.616805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.617083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] PATH_INFO: `/v3/users/4e18b7c92504430ca7b615f0ca1274b8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.631659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] RBAC: Authorizing `identity:get_user(user_id=4e18b7c92504430ca7b615f0ca1274b8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:53.633101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:53.634809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 565/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:53 2022] GET /identity/v3/users/4e18b7c92504430ca7b615f0ca1274b8 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:53.641323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:53.733751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:53.744215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5dce-ea58-43ad-8d05-2d800d564943 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:53.780269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:53.780897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:53.781176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:53.781448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:53.848532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:53.849920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:53.939717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:53.951715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 579/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 08:59:53.958156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.019444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:54.057675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f00fa-827b-49aa-a9c1-a20574580187 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.091097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.091775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.092055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.092325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] PATH_INFO: `/v3/projects/41fb59d89bc34feab7a0581ec9057967` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.093474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] RBAC: Authorizing `identity:get_project(project_id=41fb59d89bc34feab7a0581ec9057967)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.094921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d432e41-b49d-4169-bf5f-d6995254e53d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.096645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 566/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:53 2022] GET /identity/v3/projects/41fb59d89bc34feab7a0581ec9057967 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:54.102938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:54.214401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737700-b9ef-466f-a55e-809fa9f64e45 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:54.248008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:54.249188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:54.249511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:54.249831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:54.250788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=41fb59d89bc34feab7a0581ec9057967, user.id=4e18b7c92504430ca7b615f0ca1274b8) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:54.251140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:54.252617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f092205-af10-4cac-b1c5-6e7135857576 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:54.258427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 580/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 08:59:54 2022] GET /identity/v3/role_assignments?user.id=4e18b7c92504430ca7b615f0ca1274b8&scope.project.id=41fb59d89bc34feab7a0581ec9057967&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 08:59:56.351290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a966-f2bb-4734-abc7-e8b895555d93 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:56.351841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a966-f2bb-4734-abc7-e8b895555d93 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:56.351841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a966-f2bb-4734-abc7-e8b895555d93 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:56.353198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 567/1147] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:56.359464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:56.359994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:56.359994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:56.378708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:56.379933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:56.433038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:56.451706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 581/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:56.474171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0a966-f2bb-4734-abc7-e8b895555d93 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:56.583894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0a966-f2bb-4734-abc7-e8b895555d93 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:56.614983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:56.620607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:56.621253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:56.621538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:56.621814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:56.631983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:56.636912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:56.638045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 08:59:56.639242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 568/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:56.644331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:56.746315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4be29-fefb-4407-a43a-b36651da25c0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:56.781992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:56.782622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:56.782900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:56.783171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:56.783894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:56.784197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:56.785567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:56.793949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 582/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:56.800479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:56.908056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21027d4b-1f77-47fc-91d1-a39e7361e721 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:56.944975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:56.945628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:56.945939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:56.946217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] PATH_INFO: `/v3/users/4e18b7c92504430ca7b615f0ca1274b8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:56.947391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] RBAC: Authorizing `identity:get_user(user_id=4e18b7c92504430ca7b615f0ca1274b8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:56.948833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:56.950564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 569/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:56 2022] GET /identity/v3/users/4e18b7c92504430ca7b615f0ca1274b8 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:56.957164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:57.004060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:57.063659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584a48-95d4-4892-a00c-26af981e2a92 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:57.105333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:57.106033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.106313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.106584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] PATH_INFO: `/v3/projects/41fb59d89bc34feab7a0581ec9057967` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.107736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] RBAC: Authorizing `identity:get_project(project_id=41fb59d89bc34feab7a0581ec9057967)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:57.109162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf9ec0-2589-40e8-a3bb-6ea9047577ee None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:57.110859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 583/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 08:59:56 2022] GET /identity/v3/projects/41fb59d89bc34feab7a0581ec9057967 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:57.116900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:57.180326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:57.228513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba31a-1250-492a-b490-3b25625d9e37 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:57.263016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:57.264252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:57.264530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:57.264801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] PATH_INFO: `/v3/projects/41fb59d89bc34feab7a0581ec9057967/users/4e18b7c92504430ca7b615f0ca1274b8/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:57.266830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] RBAC: Authorizing `identity:create_grant(project_id=41fb59d89bc34feab7a0581ec9057967, user_id=4e18b7c92504430ca7b615f0ca1274b8, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:57.268265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f166c7-66c8-49f8-b202-e971de16ee62 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:57.282588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 570/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 08:59:57 2022] PUT /identity/v3/projects/41fb59d89bc34feab7a0581ec9057967/users/4e18b7c92504430ca7b615f0ca1274b8/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 08:59:59.285726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67c426-1a7c-4c01-96f7-2e90c36698cc None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:59.286297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67c426-1a7c-4c01-96f7-2e90c36698cc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:59.286297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67c426-1a7c-4c01-96f7-2e90c36698cc None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:59.287590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 584/1154] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 08:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 08:59:59.293134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:59.293619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:59.293730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:59.312864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 08:59:59.314104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 08:59:59.381868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 571/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 08:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 08:59:59.404784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67c426-1a7c-4c01-96f7-2e90c36698cc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:59.507513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67c426-1a7c-4c01-96f7-2e90c36698cc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:59.521144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:59.541516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:59.542220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:59.542495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:59.542761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:59.554416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 08:59:59.559619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 08:59:59.561253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 08:59:59.562167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 585/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 08:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 08:59:59.567510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:59.618143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 08:59:59.671459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041c44-5e23-41d6-af56-edc1636d8f24 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:59.705786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:59.706438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:59.706717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:59.707047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:59.707707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 08:59:59.708008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:59.709373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:59.718325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 572/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 08:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:59.724368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:59.830282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea719-bdc1-4067-8b6e-856ba34b99a3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 08:59:59.875273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 08:59:59.876302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 08:59:59.876760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 08:59:59.877129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] PATH_INFO: `/v3/users/4e18b7c92504430ca7b615f0ca1274b8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 08:59:59.878341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] RBAC: Authorizing `identity:get_user(user_id=4e18b7c92504430ca7b615f0ca1274b8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 08:59:59.879792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 08:59:59.881442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 586/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 08:59:59 2022] GET /identity/v3/users/4e18b7c92504430ca7b615f0ca1274b8 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 08:59:59.887486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 08:59:59.999690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddffac1-819f-4d37-933d-97ce62922052 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:00.013096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:00.035246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:00.035874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:00.036183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:00.036454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:00.103642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:00.105029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:00.211286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 573/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 08:59:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:00:00.218020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:00.262509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:00.328475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9bf9-ca6b-4a3f-8ffa-885016c74a26 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:00.362600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:00.363158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:00.363432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:00.363700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] PATH_INFO: `/v3/projects/41fb59d89bc34feab7a0581ec9057967` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:00.364958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] RBAC: Authorizing `identity:get_project(project_id=41fb59d89bc34feab7a0581ec9057967)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:00.366512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55e902-4f2f-479b-8e25-802ed31ac107 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:00.368200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 587/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 09:00:00 2022] GET /identity/v3/projects/41fb59d89bc34feab7a0581ec9057967 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:00.374947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:00.389621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:00.475596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8c6cd-87a2-4bb8-8964-8e9467985306 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:00.510759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:00.511385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:00.511666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:00.511941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:00.512732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=41fb59d89bc34feab7a0581ec9057967, user.id=4e18b7c92504430ca7b615f0ca1274b8) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:00.513044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:00.515092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50170d-5d5c-4ec4-83b7-338ecf778c39 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:00.521266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 574/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:00:00 2022] GET /identity/v3/role_assignments?user.id=4e18b7c92504430ca7b615f0ca1274b8&scope.project.id=41fb59d89bc34feab7a0581ec9057967&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:02.564614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41df0-5015-4d78-9709-2aa5c631ad56 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:02.565213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41df0-5015-4d78-9709-2aa5c631ad56 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:02.565213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd41df0-5015-4d78-9709-2aa5c631ad56 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:02.566727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 588/1162] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:02.572314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:02.572870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:02.572870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:02.592343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:02.593565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:02.665091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 575/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:02.688226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd41df0-5015-4d78-9709-2aa5c631ad56 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:02.790091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd41df0-5015-4d78-9709-2aa5c631ad56 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:02.801100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:02.824268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:02.824819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:02.825096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:02.825392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:02.892199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:02.893666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:03.005390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 589/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:00:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:00:03.015448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:03.028353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:03.125253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30f6b5-dcbe-48bf-a59d-82cdf5c9e74f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:03.160067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:03.160708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:03.160984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:03.161251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:03.168242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:03.169737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c086ff0e-858a-4e20-a0ed-a422ee32bbbb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:00:03.181282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:00:03.182949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:00:03.184059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 576/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 09:00:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:00:03.188903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:03.200769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:03.289579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f4fd-2245-47c0-94a2-21ca2c0b1577 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:03.331635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:03.332626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:03.333084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:03.333516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:03.334634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:03.335124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:03.337501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f03f-d678-4af1-ad77-4efcd7bf087f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:03.348600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 590/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Jul 28 09:00:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:05.366209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ca41-b9d5-41cc-bf7c-7f686ab35d8c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:05.366782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ca41-b9d5-41cc-bf7c-7f686ab35d8c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:05.366782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ca41-b9d5-41cc-bf7c-7f686ab35d8c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:05.368111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 577/1167] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:05.374042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12727-7e25-4062-a5d7-bed16d9c0ada None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:05.374516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12727-7e25-4062-a5d7-bed16d9c0ada None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:05.374617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12727-7e25-4062-a5d7-bed16d9c0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:05.393222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a8f12727-7e25-4062-a5d7-bed16d9c0ada None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:05.394469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a8f12727-7e25-4062-a5d7-bed16d9c0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:05.461593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 591/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:05.484242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ca41-b9d5-41cc-bf7c-7f686ab35d8c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:05.551471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:05.591001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ca41-b9d5-41cc-bf7c-7f686ab35d8c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:05.634400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:05.634943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:05.635221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:05.635493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:05.636209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:05.637580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b391-d366-45a0-83da-25ced5c37d12 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:05.654557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 578/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Thu Jul 28 09:00:05 2022] POST /identity/v3/domains => generated 267 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:00:07.702969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4888-ca25-4973-a840-767eb4cadfd4 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:07.703532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4888-ca25-4973-a840-767eb4cadfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:07.703532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4888-ca25-4973-a840-767eb4cadfd4 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:07.704862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 592/1170] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:07.710615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:07.711106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:07.711207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:07.729611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:07.730830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:07.797529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 579/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:07.821040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c4888-ca25-4973-a840-767eb4cadfd4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:07.825267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:07.971680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c4888-ca25-4973-a840-767eb4cadfd4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:08.005231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:08.005894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:08.006177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:08.006446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] PATH_INFO: `/v3/domains/eb1f9ff5593a4837af860add6d72a100` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:08.017696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb1f9ff5593a4837af860add6d72a100)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:08.019190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277251a9-081f-44a1-bcb6-5ee4115803d5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:08.020897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 593/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Jul 28 09:00:07 2022] GET /identity/v3/domains/eb1f9ff5593a4837af860add6d72a100 => generated 267 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:08.026826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:08.084790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:08.135526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eae8cb-8a91-4bf0-b783-ae9fdf63dab6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:08.170381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:08.171012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:08.171295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:08.171574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:08.172256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:08.173710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b192df-8cb6-4f26-98eb-47d37b3e5b73 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:08.204231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 580/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 09:00:08 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:00:10.251608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75394f8f-c7f8-46bb-9793-b2f16b93d1f6 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:10.252123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75394f8f-c7f8-46bb-9793-b2f16b93d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:10.252123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75394f8f-c7f8-46bb-9793-b2f16b93d1f6 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:10.253449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 594/1174] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:00:10.259132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:10.259635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:10.259736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:10.277531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:10.278763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:10.345959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 581/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:10.371185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75394f8f-c7f8-46bb-9793-b2f16b93d1f6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:10.435407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:10.476231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75394f8f-c7f8-46bb-9793-b2f16b93d1f6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:10.515262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:10.515831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:10.516111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:10.516386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] PATH_INFO: `/v3/domains/eb1f9ff5593a4837af860add6d72a100` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:10.517556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb1f9ff5593a4837af860add6d72a100)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:10.519037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af93ef9-58b7-4817-83d6-e88d24fe38fd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:10.520652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 595/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Jul 28 09:00:10 2022] GET /identity/v3/domains/eb1f9ff5593a4837af860add6d72a100 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:10.526511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:10.567765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:10.637532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a556-93e9-46be-b3c8-e179a085645c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:10.674619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:10.675233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:10.675510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:10.675781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:10.676462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:10.677948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2dbe1-5a76-4a57-8dbc-6ed965c63cf9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:10.699203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 582/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 09:00:10 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:00:12.775587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944e38-defb-4c97-8c88-5b0ff0742640 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:12.776176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944e38-defb-4c97-8c88-5b0ff0742640 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:12.776176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944e38-defb-4c97-8c88-5b0ff0742640 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:12.777492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 596/1178] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:12.782824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:12.783284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:12.783389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:12.802203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:12.803418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:12.876763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 583/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:12.900199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05944e38-defb-4c97-8c88-5b0ff0742640 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.008536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05944e38-defb-4c97-8c88-5b0ff0742640 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.043342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:13.049407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.050081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.050358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.050636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.061960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:00:13.066915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:00:13.068048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:00:13.074054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 597/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:00:13.074959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.105591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:13.177519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2372-f9ad-466b-b13c-0c6bd51814d1 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.211094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.211718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.211995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.212265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.212980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:13.213281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:13.214684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:13.223933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 584/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:00:13.229873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.336155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcc019-4721-4a67-80e5-d8ca70f3b903 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.369557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.370200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.370472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.370742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] PATH_INFO: `/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.384210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff94ce0ee7c4ddba1cb5acd910123ce)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:13.385652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:13.387379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 598/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 09:00:13 2022] GET /identity/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:13.393824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.478723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:13.493694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714924c7-1180-4e36-933f-e1b3d2665476 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.527264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.527885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.528167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.528439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.597047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:13.598464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:13.681585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:13.697566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 585/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:00:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:00:13.704222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.757426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:13.807203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00b8c-ddb3-4f7d-95eb-b546d46e4f34 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.844382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.845029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.845309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.845582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] PATH_INFO: `/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.846759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] RBAC: Authorizing `identity:get_project(project_id=e47c59f2c4ec43a0abb0b565db393d2a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:13.848269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c71b8c-7bb0-433d-9578-419cd6ccfeca None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:13.850080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 599/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 09:00:13 2022] GET /identity/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:13.858910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:13.964144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cae549-f7c2-4d05-bbe4-22736f9722f8 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:13.997323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:13.998489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:13.998771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:13.999039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:13.999817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=e47c59f2c4ec43a0abb0b565db393d2a, user.id=1ff94ce0ee7c4ddba1cb5acd910123ce) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:14.000123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:14.001554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28f9e-43fe-480d-bb63-840352446afa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:14.009240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 586/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:00:13 2022] GET /identity/v3/role_assignments?user.id=1ff94ce0ee7c4ddba1cb5acd910123ce&scope.project.id=e47c59f2c4ec43a0abb0b565db393d2a&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:15.979134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313d2ef-25d3-4d72-8c99-3982210adc81 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:15.979647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313d2ef-25d3-4d72-8c99-3982210adc81 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:15.979738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313d2ef-25d3-4d72-8c99-3982210adc81 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:15.981062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 600/1186] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:15.986824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:15.987244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:15.987386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.005663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:16.006892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:16.051406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:16.071943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 587/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:16.094432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313d2ef-25d3-4d72-8c99-3982210adc81 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:16.191466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313d2ef-25d3-4d72-8c99-3982210adc81 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:16.207950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:16.224716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:16.225348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:16.225623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:16.225919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.236098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:00:16.240944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:00:16.242070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:00:16.242919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 601/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:00:16.248228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:16.344192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3638226-87d9-4dc1-9a20-3e144cd5dc62 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:16.377154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:16.377771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:16.378082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:16.378359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.379068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:16.379374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:16.380739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:16.389126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 588/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:16.395094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:16.491892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e74b57-91c0-40f5-8b78-068ef332171c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:16.524549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:16.525184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:16.525475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:16.525753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] PATH_INFO: `/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.526939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] RBAC: Authorizing `identity:get_user(user_id=1ff94ce0ee7c4ddba1cb5acd910123ce)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:16.528369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:16.530556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 602/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 09:00:16 2022] GET /identity/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:16.536458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:16.579064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:16.634440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b667ff4-2ec1-4e1b-9684-8e54dec05292 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:16.667612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:16.668244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:16.668522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:16.668794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] PATH_INFO: `/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.669952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] RBAC: Authorizing `identity:get_project(project_id=e47c59f2c4ec43a0abb0b565db393d2a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:16.671381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c7cf-034d-4271-bbcc-1ccdd7aa2df2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:16.673023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 589/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 09:00:16 2022] GET /identity/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:16.679013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:16.724301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:16.777353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115ebf-c196-40cd-961b-a62363f3d06b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:16.809230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:16.809909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:16.810723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:16.810996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] PATH_INFO: `/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a/users/1ff94ce0ee7c4ddba1cb5acd910123ce/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:16.812887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] RBAC: Authorizing `identity:create_grant(project_id=e47c59f2c4ec43a0abb0b565db393d2a, user_id=1ff94ce0ee7c4ddba1cb5acd910123ce, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:16.814363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc71c7-7df5-4f45-862a-9eaf48f74138 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:16.827078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 603/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 09:00:16 2022] PUT /identity/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a/users/1ff94ce0ee7c4ddba1cb5acd910123ce/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:00:18.799292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ef00e-b6c8-4092-be58-38abee316673 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:18.799828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ef00e-b6c8-4092-be58-38abee316673 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:18.799828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ef00e-b6c8-4092-be58-38abee316673 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:18.801187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 590/1193] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:00:18.806194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:18.806580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:18.806758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:18.825865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:00:18.827066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:00:18.891061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 604/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:00:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:00:18.913973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ef00e-b6c8-4092-be58-38abee316673 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.023427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ef00e-b6c8-4092-be58-38abee316673 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.034728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:19.059112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.059668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.059952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.060390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.074566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:00:19.080670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:00:19.081811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:00:19.082748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 591/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:00:19.088042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.122252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:19.194554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76593e26-a8eb-451c-bf75-4a74ab1c322d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.227008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.227618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.227892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.228160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.228869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:19.229169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:19.230673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:19.239012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 605/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:19.244785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.353306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8bee3-0c4d-462f-96d4-f7c95dc92d22 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.391311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.391950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.392230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.392506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] PATH_INFO: `/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.393662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff94ce0ee7c4ddba1cb5acd910123ce)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:19.395138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:19.396759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 592/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Jul 28 09:00:19 2022] GET /identity/v3/users/1ff94ce0ee7c4ddba1cb5acd910123ce => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:19.402571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.494790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:19.499105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c348eb6-c565-43ca-81dd-928fabfa20ee None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.531907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.532528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.532812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.533089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.598510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:19.599959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:19.698296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:19.705877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 606/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:00:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:00:19.708874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.749408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:19.810164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb3cf-39aa-4ba8-8d15-087c747003d2 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.843498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.844129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.844430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.844703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] PATH_INFO: `/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.845871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e47c59f2c4ec43a0abb0b565db393d2a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:19.847326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df5779-49d7-44e7-96ee-9a74dadd39f8 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:19.848984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 593/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Jul 28 09:00:19 2022] GET /identity/v3/projects/e47c59f2c4ec43a0abb0b565db393d2a => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:19.855064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:00:19.952522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114b4d8-986d-4944-831c-44273c790b4c None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:00:19.986379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:00:19.986983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:00:19.987257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:00:19.987524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:00:19.988303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=e47c59f2c4ec43a0abb0b565db393d2a, user.id=1ff94ce0ee7c4ddba1cb5acd910123ce) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:00:19.988605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:00:19.990040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716eb4-8b05-41ee-8d47-6486d6d5cc5d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:00:19.996046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 607/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:00:19 2022] GET /identity/v3/role_assignments?user.id=1ff94ce0ee7c4ddba1cb5acd910123ce&scope.project.id=e47c59f2c4ec43a0abb0b565db393d2a&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:00:29.839237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=47 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:00:29.993396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=69 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:15.655802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705141c-9cab-4802-91c4-254000317d87 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:15.656398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705141c-9cab-4802-91c4-254000317d87 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:15.656398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705141c-9cab-4802-91c4-254000317d87 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:15.657857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 594/1201] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:01:15.664330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:15.664829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:15.664829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:15.688481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:15.689794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:15.779657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 608/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:15.806136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705141c-9cab-4802-91c4-254000317d87 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:15.952396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705141c-9cab-4802-91c4-254000317d87 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:15.994064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:15.995032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:15.995320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:15.996299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:16.008788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:16.010463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:16.028805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:16.029857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:16.030839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:16.030839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:16.030839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:16.030839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:16.030839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 595/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:16.036479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:16.166475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34c265-96f9-41a3-ab10-f2846df5c73f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:16.203140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:16.211231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:16.212077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:16.212330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:16.212765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:16.213465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:16.215567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:16.217435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5c1dc-0dcc-40ac-9964-7e6ac5c639a7 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:16.231499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 609/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:16.238100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:16.323906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:16.387155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86bde-3de0-4191-b2e5-8c412d6930e9 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:16.429476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:16.430180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:16.430401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:16.430675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:16.431346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:16.432778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62498142-b9ff-49c4-a682-9e16f7b54c6c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:16.455704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 596/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Jul 28 09:01:16 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:01:18.760271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c94d-b5d0-4224-82db-0189b3e1e531 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:18.760818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c94d-b5d0-4224-82db-0189b3e1e531 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:18.760818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94c94d-b5d0-4224-82db-0189b3e1e531 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:18.762477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 610/1206] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:18.768488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:18.768995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:18.769095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:18.788439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:18.789699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:18.877270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 597/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:18.900955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94c94d-b5d0-4224-82db-0189b3e1e531 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:19.039129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94c94d-b5d0-4224-82db-0189b3e1e531 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:19.083641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:19.084297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:19.084623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:19.084838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:19.099295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:19.105619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:19.106690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:19.107530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 611/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 09:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:19.114054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:19.162135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:19.231203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05144cab-1ee8-4049-883b-68a17c206ee9 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:19.269051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:19.269601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:19.269912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:19.270191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:19.270936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:19.271241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:19.272621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:19.285629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 598/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 09:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:19.292108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:19.375841 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:19.464351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd139-8ebc-4af4-b90b-6efe541a3a91 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:19.510537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:19.512102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:19.512102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:19.512102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:19.525763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:19.527301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:19.543213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:19.544882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:19.545784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:19.545784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:19.545784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:19.545784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:19.545784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 612/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:19.555661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:19.679510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:19.683802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94117c0-828b-468e-b0fa-3deb422f306c None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:19.728462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:19.729140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:19.729565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:19.729867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:19.730648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:19.730935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:19.732885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:19.746243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 599/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:19.752515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:19.875574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4cc4f-4744-4012-b6d9-175c71199c2b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:19.877189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:19.930252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:19.931372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:19.933229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:19.933229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:19.941005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:19.942626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:19.955952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:19.957023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:19.958028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 613/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:19.965263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:20.106303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8cdf1-0bcd-4569-b1e3-db0bc12af985 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:20.153063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:20.154653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.154934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.155202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.156006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:20.156304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:20.158356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:20.179500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 600/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:20.186288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:20.354218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef683f-1536-4d02-b559-dc6270ed73e5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:20.410756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:20.411898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.412474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.412869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.459311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:20.513342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:20.515211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:20.667567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 614/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:01:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:01:20.673404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:20.695180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:20.784427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b9487-7eec-4de2-8561-af8762984ffc None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:20.828761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:20.829431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:20.829720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:20.830028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:20.841432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:20.842973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:20.860017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:20.860995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:20.861910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:20.861910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:20.861910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:20.861910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:20.861910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 601/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:20.870227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:20.946270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:20.992723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4f75-1802-407b-a293-84d69f145ead None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:21.036917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:21.037556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:21.037858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:21.038139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:21.038866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:21.039160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:21.040621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:21.054128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 615/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:21.060918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:21.151131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:21.192084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1da5b-69a9-4b4c-9ad9-d136b30bec67 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:21.238295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:21.238992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:21.239302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:21.239548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:21.248076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:21.249581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:21.262998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:21.264029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:21.265084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 602/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:01:21.270376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:21.401476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd36a3-f7da-4d1d-a9e7-a0f9f44fecfe None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:21.428425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:21.445896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:21.446656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:21.447074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:21.447435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:21.448444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:21.448825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:21.450656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d993-c5e8-4634-9771-5dae131996f1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:21.468452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 616/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:21.474757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:21.594796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280e94-640c-4901-b1ac-758c0519bb16 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:21.648190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:21.659261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:21.659999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:21.660287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:21.660570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:21.661536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=4ad9d639d97b415aba4f3772e8ea0f2d) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:21.661775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:21.663345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808cd9-6e49-46cd-806c-14ffeb86c358 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:21.670056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 603/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:01:21 2022] GET /identity/v3/role_assignments?user.id=4ad9d639d97b415aba4f3772e8ea0f2d&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:23.791019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8da7-f5e3-488a-b558-03472fc9c537 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:23.791579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8da7-f5e3-488a-b558-03472fc9c537 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:23.791579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8da7-f5e3-488a-b558-03472fc9c537 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:23.792980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 617/1220] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:23.799491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:23.800041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:23.800041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:23.818751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:23.819986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:23.898104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 604/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:23.922994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f8da7-f5e3-488a-b558-03472fc9c537 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.050193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f8da7-f5e3-488a-b558-03472fc9c537 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:24.093990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:24.094789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:24.095230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:24.095547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:24.108600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:24.115397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:24.116604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:24.117832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 618/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 09:01:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:01:24.124251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.235526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3f42d-a065-4f0d-bf1b-1a43595cb746 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:24.263263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:24.274672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:24.275303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:24.275631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:24.275900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:24.276634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] RBAC: Adding query filter params (name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:24.276935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:24.278442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:24.287317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 605/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 09:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:24.293390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.344976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:24.419322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4dedc-6234-4e75-8823-b8cd46c2e04f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:24.459481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:24.460232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:24.460522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:24.460830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:24.472453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:24.473977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:24.489556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:24.492617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:24.493499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:24.493499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:24.493499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:24.493499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:24.493499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 619/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:24.499822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.642137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985e11-5f1a-4b1d-9068-d9b3e1e01819 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:24.682502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:24.683116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:24.683397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:24.683940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:24.685212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:24.685522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:24.686951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:24.698894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 606/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:24.704738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.810382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:24.817338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284b52-efc0-4cd7-9175-e01afac3e570 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:24.853769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:24.854441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:24.854772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:24.855135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] PATH_INFO: `/v3/users/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:24.864365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:24.866279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:24.880512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:24.881719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:24.883313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 620/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:24.888061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:24.950335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:25.013983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cd63d-0fb5-4d6a-b5df-809b34139bb6 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:25.054288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:25.054918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.055210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.055486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.056284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:25.056776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:25.058253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:25.078517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 607/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:25.084808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.209482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb671f5-8832-4cbe-9c8f-0dd8f021c4db None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:25.255806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:25.256583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.256888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.257169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.269921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:25.271638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:25.287197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:25.293398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:25.295063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:25.295063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:25.295063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:25.295063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:25.295063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 621/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:25.295351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.416870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55f95-685f-41e6-8f89-e40bf349d7fa None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:25.419216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:25.457233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:25.457953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.458208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.458457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.459164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:25.459443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:25.460879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:25.473380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 608/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:25.479338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.489602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:25.611290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7141b6-e39b-4404-a124-2cb4c60baacc None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:25.654218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:25.655093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.655511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.655644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.662902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:25.664402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:25.676978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:25.678354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:25.679328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 622/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:25.684571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.810881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766a52a-43be-45bb-bcae-a90fb170e79e None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:25.848182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:25.848829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:25.849084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:25.849361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:25.850290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:25.850600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:25.852036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022b8f1-a7fd-4bff-be8c-8419833e2520 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:25.864521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 609/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:25.872753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:25.950936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:26.003867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3f2c0-852c-4127-84ae-237e0baf7ef5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:26.051348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:26.052668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:26.053155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:26.053485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/4ad9d639d97b415aba4f3772e8ea0f2d/roles/df7539c1d8d5416ea48e8bbd1e442a53` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:26.073079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=4ad9d639d97b415aba4f3772e8ea0f2d, role_id=df7539c1d8d5416ea48e8bbd1e442a53)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:26.076247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f9af0-9ffe-4529-b799-d6e438e33e69 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:26.095643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 623/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 09:01:25 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/4ad9d639d97b415aba4f3772e8ea0f2d/roles/df7539c1d8d5416ea48e8bbd1e442a53 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:01:28.304660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d426f-2402-4114-8282-ffb8c2ec3d8a None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.305201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d426f-2402-4114-8282-ffb8c2ec3d8a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.305201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d426f-2402-4114-8282-ffb8c2ec3d8a None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.306603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 610/1233] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:28.320318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.320833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.320950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.340018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:28.341318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:28.433979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 624/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:28.455543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d426f-2402-4114-8282-ffb8c2ec3d8a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:28.496113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:28.601004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d426f-2402-4114-8282-ffb8c2ec3d8a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:28.652231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:28.652899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.653242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.653474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.664115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:28.670251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:28.672149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:28.673790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 611/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Jul 28 09:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:28.679989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:28.734097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:28.793405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba94a-5b1d-4354-9fef-1b0437985ac8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:28.829274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:28.829943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:28.830224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:28.830520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:28.831242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:28.831611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:28.832942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:28.844474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 625/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Jul 28 09:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:28.852870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:28.972219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:29.008656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0ea79-daaa-4a3e-8766-9359266e4036 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:29.054007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:29.054661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:29.054942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:29.055218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:29.066932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.068375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:29.086430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:29.087561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:29.088468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:29.088468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:29.088468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:29.088468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:29.088468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 612/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:29.093746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:29.210684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86635896-f7a9-4a89-8f54-2a8b375f69cd None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:29.222482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:29.252655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:29.253413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:29.253686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:29.253994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:29.254710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:29.255038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.256481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:29.270901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 626/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:29.277728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:29.391535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440d93f-be98-4d4a-9b9e-4a9c1781f7f5 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:29.431617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:29.432273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:29.432548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:29.432819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:29.436746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:29.443313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.445199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:29.457860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:29.459062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:29.460020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 613/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:29.465723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:29.596544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcbdaa-ced3-49e7-b63e-4d78365f2e8d None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:29.641550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:29.642316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:29.642671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:29.643053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:29.644363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:29.644769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.646828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:29.670126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 627/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:29.677392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:29.794634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb25c-f902-4a9c-9129-fef15519eec1 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:29.838170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:29.838792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:29.839071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:29.839369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:29.928403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:29.937319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:29.938931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:30.089219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 614/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:01:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 09:01:30.095603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:30.129257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:30.222300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf15452-0122-47c3-b77d-1b3969a05ced None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:30.276486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:30.277221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:30.277603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:30.277978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:30.293254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:30.294794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:30.315226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:30.316528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:30.323165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:30.323165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:30.323165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:30.323165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:30.323165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 628/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:30.323504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:30.424485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:30.433347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b35d92-50c5-4447-895d-0a07eae29379 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:30.475865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:30.476584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:30.476931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:30.477272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:30.478070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:30.478445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:30.479899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:30.493046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 615/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:30.500605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:30.609264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:30.636206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369c47-513f-4851-92f4-2277717750ef None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:30.676003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:30.676590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:30.676872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:30.677151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:30.686376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:30.688041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:30.701581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:30.702778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:30.703898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 629/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:30.709552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:30.829037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24081400-a7fb-4dd6-8c49-5faea4a5fb5d None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:30.874774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:30.875440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:30.875744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:30.876023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:30.876783 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:30.877092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:30.879935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d71d5-2512-4cc8-b7c1-ee6dd0c11b28 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:30.894403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 616/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:30.901385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:31.038166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb092-f3ad-46c6-afd2-550285651465 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:31.077123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:31.077753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:31.078064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:31.078339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:31.079169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] RBAC: Adding query filter params (role.id=df7539c1d8d5416ea48e8bbd1e442a53, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=4ad9d639d97b415aba4f3772e8ea0f2d) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:31.079486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:31.081023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d340-c371-4ef2-ad35-ebbcae695a02 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:31.089690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 630/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:01:30 2022] GET /identity/v3/role_assignments?user.id=4ad9d639d97b415aba4f3772e8ea0f2d&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=df7539c1d8d5416ea48e8bbd1e442a53 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:33.243417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361ead1-1c4f-43df-90d3-48ecbe3514fc None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:33.243990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361ead1-1c4f-43df-90d3-48ecbe3514fc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:33.243990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361ead1-1c4f-43df-90d3-48ecbe3514fc None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:33.245214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 617/1247] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:33.251495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:33.251985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:33.252074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:33.259210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:33.272561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:33.273804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:33.350332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 631/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:33.374769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361ead1-1c4f-43df-90d3-48ecbe3514fc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:33.398132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:33.499709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361ead1-1c4f-43df-90d3-48ecbe3514fc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:33.542877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:33.543628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:33.544208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:33.544466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:33.561562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:33.567749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:33.568738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:33.569738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 618/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:33.576595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:33.705859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0881e-38a5-4625-986f-8c10fad526f7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:33.743459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:33.751536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:33.752502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:33.752901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:33.753355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:33.754547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:33.755103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:33.757808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:33.774408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 632/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:33.782773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:33.879573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:33.919780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120c36-54cd-4f45-8818-51e90a324b1b None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:33.966300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:33.966951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:33.967231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:33.967613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:33.979616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:33.981119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:33.997805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:33.999260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:34.000379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:34.000379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:34.000379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:34.000379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:34.000379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 619/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:34.005329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:34.137667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5203-1447-4f6c-adbe-d6491441b0b9 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:34.175288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:34.175921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.176197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.176466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.177174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:34.177466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:34.178880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:34.197630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 633/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:34.203587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:34.329881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab73df-fe64-4949-a433-02896284c0d0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:34.367798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:34.377076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:34.377868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.378152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.378425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.386677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:34.388689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:34.401112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:34.402831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:34.404363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 620/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 09:01:34.409044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:34.486808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:34.541124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5f205-3a82-49f4-b4ed-1fd05a1debbb None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:34.584823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:34.585421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.585896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.586262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.587528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:34.588453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:34.590504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:34.624671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 634/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:34.625964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:34.759252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da35e-137c-4d8c-bcd4-9224db76d471 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:34.804638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:34.805492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:34.805883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:34.806169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:34.876078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:34.892926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:34.896105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:35.013135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 621/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:01:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:01:35.019833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:35.169898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334a09-cc88-48d9-a660-52719704fc7b None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:35.187572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:35.218461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:35.219108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:35.219457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:35.219678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:35.234711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:35.236351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:35.259190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:35.260475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:35.261343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:35.261343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:35.261343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:35.261343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:35.261902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 635/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:35.268350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:35.378175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:35.395823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ced2-2c00-4b41-80eb-288e7b372661 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:35.443733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:35.443733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:35.444048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:35.444283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:35.445054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:35.445361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:35.446811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:35.465259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 622/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:35.471499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:35.594667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85a30d-f2eb-4b9c-9272-328a3edca50a None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:35.643626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:35.644749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:35.645235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:35.645550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:35.654116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:35.655712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:35.676598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:35.677798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:35.678842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 636/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:35.685101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:35.796199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090508-aa0e-47dc-8350-12c7387fd4ac None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:35.834914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:35.835677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:35.835996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:35.836273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:35.837138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:35.837490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:35.839158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dbf4e-0ed3-4050-b844-f1d4269c9945 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:35.854305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 623/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:35.861230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:35.889331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:35.988412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5830-044e-4464-b7ab-a1458c199ef1 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:36.030117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:36.030861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:36.031223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:36.031462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:36.032273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=4ad9d639d97b415aba4f3772e8ea0f2d) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:36.032584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:36.034094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc728-b08f-49d9-af86-57985a9963ed None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:36.044490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 637/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:01:35 2022] GET /identity/v3/role_assignments?user.id=4ad9d639d97b415aba4f3772e8ea0f2d&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:01:38.224082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934559-db2f-4612-befd-8cdc6f875c1c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.224593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934559-db2f-4612-befd-8cdc6f875c1c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.224593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d934559-db2f-4612-befd-8cdc6f875c1c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.226100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 624/1261] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:38.232060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.232502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.232613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.251046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:38.252224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:38.322674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 638/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:38.344132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d934559-db2f-4612-befd-8cdc6f875c1c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:38.355140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:38.443314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d934559-db2f-4612-befd-8cdc6f875c1c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:38.476177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:38.476717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.476996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.477270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.488246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:38.493195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:38.494478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:38.495591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 625/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:38.503221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:38.521498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:38.610890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39dd8-653e-4d0e-8f9c-adfdd0afac89 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:38.646756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:38.647385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.647663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.647933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.648646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:38.648948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:38.650340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:38.659199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 639/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:38.665007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:38.735067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:38.768683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dbb74-034e-47d8-8b68-2c48879ed321 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:38.806497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:38.807136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.807418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.807691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.818126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:38.819597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:38.834582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:38.835719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:38.836702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:38.836702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:38.836702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:38.836702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:38.836702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 626/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:38.843171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:38.930643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:38.951416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89825045-6f49-4184-884e-a5df01655d60 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:38.984427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:38.985046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:38.985380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:38.985597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:38.986317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:38.986622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:38.988103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.000036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 640/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:39.005733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.106380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4b30b-c370-4918-8392-751c4d0a6d40 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.125816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:39.140559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.141196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.141478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.141781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.149381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.150850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:39.162498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:39.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:39.169322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 627/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:39.170170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.267380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ec13-dabd-4e56-a961-3f409d87f9f3 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.300100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.300722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.301008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.301285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.302112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:39.302423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.303904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.313181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:39.322322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 641/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:39.329096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.425887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5627c-dd31-4bd3-8b0a-df15c9ca58b7 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.459211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.459949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.460271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.460589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.471449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.472916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:39.487306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:39.488353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:39.489309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:39.489309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:39.489309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:39.489309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:39.489309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 628/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:39.494366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.604781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042158-0162-4cf2-b338-8c64ff1208d6 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.645330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.646092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.646386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.646663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.647442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:39.647768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.649256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.661407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 642/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:39.667114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.699474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:39.769150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c987ef-1568-4a15-8766-551f3aa88415 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.803702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.804340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.804620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.804895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.812693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.814180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:39.825571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:39.826673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:39.827738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 629/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:39.833140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:39.892500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:39.940492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7460-f637-4813-8ad6-eae6722c47d5 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:39.981859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:39.982503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:39.982780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:39.983051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:39.983807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:39.984111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:39.985543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c3a6b-2300-40a8-a817-3693695fa722 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:39.999351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 643/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:40.005389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:40.099292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:40.117977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e530-c713-4ab3-b98e-97b520671f6f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:40.160077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:40.160664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:40.160946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:40.161226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] PATH_INFO: `/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/4ad9d639d97b415aba4f3772e8ea0f2d/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:40.163214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] RBAC: Authorizing `identity:create_grant(project_id=65cbf330066d4039af6a9cacf70f36f4, user_id=4ad9d639d97b415aba4f3772e8ea0f2d, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:40.164661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac888a71-5d6a-42cd-a3d0-13dbabbc08fd None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:40.179348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 630/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Jul 28 09:01:40 2022] PUT /identity/v3/projects/65cbf330066d4039af6a9cacf70f36f4/users/4ad9d639d97b415aba4f3772e8ea0f2d/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:01:42.180228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4350-bd3a-40ae-bb1a-48c521d348c2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.180748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4350-bd3a-40ae-bb1a-48c521d348c2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.180748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4350-bd3a-40ae-bb1a-48c521d348c2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.182144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 644/1274] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:42.188508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.188991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.189080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.207594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:42.208787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:42.276323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 631/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:42.301367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4350-bd3a-40ae-bb1a-48c521d348c2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:42.410680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4350-bd3a-40ae-bb1a-48c521d348c2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:42.428915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:42.446186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:42.446813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.447034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.447307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.457615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 09:01:42.462614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:42.463754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:42.464952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 645/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:42.470497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:42.551864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:42.575886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118401e4-8cc1-49f3-80d1-02de2eebac5b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:42.621545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:42.622743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.622889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.623103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.624017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:42.624390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:42.626286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:42.642371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 632/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:42.652385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:42.758210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa576a-3a6a-4573-a9bc-fa422d4eefe8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:42.792930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:42.793565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.793905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.794182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.803973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:42.805514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:42.821005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:42.822089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:42.823332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:42.823332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:42.823332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:42.823332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:42.823332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 646/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:42.828509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:42.926545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575a382-3eae-49df-af40-1d935dbbeb83 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:42.960472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:42.961100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:42.961383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:42.961661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:42.962397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:42.962707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:42.964110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:42.977359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 633/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:42.983060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:43.014696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:43.085703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179da4a-fdeb-4bfe-9de5-4ea93952ee63 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:43.122463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:43.123112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:43.123391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:43.123664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:43.131518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:43.132956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 09:01:43.144943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:43.146150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:43.147321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 647/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:43.152058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:43.155971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:43.253806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4f48-d36f-4ef2-b6fe-a46364e8efd4 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:43.288046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:43.288597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:43.288880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:43.289154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:43.289994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:43.290305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:43.291746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:43.310517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 634/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Jul 28 09:01:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:43.316689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:43.408171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:43.418152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b49d59-934a-434b-b9d4-cc6a086adcc0 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:43.453234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:43.453883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:43.454168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:43.454444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:43.522490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:43.523941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:43.625858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:43.636369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 648/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Jul 28 09:01:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 09:01:43.642467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:43.703501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:43.745133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035a64-4a68-4606-940e-7eab710abac0 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:43.781560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:43.782240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:43.782522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:43.782795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:43.793210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:43.795298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 09:01:43.810730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:43.811842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:43.812989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 09:01:43.812989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 09:01:43.812989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 09:01:43.812989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:43.812989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 635/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:43.817749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:43.915302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ac1e-1a51-4dee-89cf-5c852d1ab254 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:43.951224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:43.951842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:43.952124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:43.952398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:43.953096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:43.953401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:43.954805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:43.967432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 649/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Jul 28 09:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:43.973091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:44.070330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53548f-d96d-4499-a6c0-cc310b45db0f None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:44.083864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:44.109141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:44.109788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:44.110100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:44.110376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:44.117115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:44.118584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 09:01:44.129929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:44.131083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: ERROR keystone.server.flask.application Jul 28 09:01:44.132235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 636/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Jul 28 09:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:44.137000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:44.193340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:44.244664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ad4b0-8821-42ea-a95f-8a7c397df17f None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:44.279662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:44.280295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:44.280576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:44.280853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:44.281607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:44.281945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:44.283382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe8a69-670e-4562-be8d-b1c939d389f6 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:44.296343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 650/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Jul 28 09:01:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:44.302665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:44.417346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44225283-ff17-4b58-a550-f17a06110fef None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:44.453106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:44.453635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:44.453949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:44.454225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:44.455016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] RBAC: Adding query filter params (role.id=6f1ebd186c9845f39f6451326a6fcbac, scope.project.id=65cbf330066d4039af6a9cacf70f36f4, user.id=4ad9d639d97b415aba4f3772e8ea0f2d) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:44.455328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:44.456762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080617f1-9abc-4fed-ab15-0150253d7e04 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:44.462800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 637/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Jul 28 09:01:44 2022] GET /identity/v3/role_assignments?user.id=4ad9d639d97b415aba4f3772e8ea0f2d&scope.project.id=65cbf330066d4039af6a9cacf70f36f4&role.id=6f1ebd186c9845f39f6451326a6fcbac => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:46.554427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8705-37c7-463f-b14c-d69ba3478202 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:46.554969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8705-37c7-463f-b14c-d69ba3478202 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:46.554969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8705-37c7-463f-b14c-d69ba3478202 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:46.556346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 651/1288] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:46.562094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:46.562593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:46.562793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:46.582603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:46.583865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:46.624064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:46.677113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 638/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:46.699570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8705-37c7-463f-b14c-d69ba3478202 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:46.800979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8705-37c7-463f-b14c-d69ba3478202 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:46.813924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:46.838851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:46.840459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:46.840738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:46.841010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:46.841646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:46.843064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:46.851972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:46.853184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:46.854474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 652/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 09:01:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:46.859233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:46.957524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bd2d2-cb28-4bbe-9e28-5db6dbf69ebe None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:46.990063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:46.990687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:46.990973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:46.991248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:46.991921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:46.992226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:46.993595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c06083-81da-4789-9221-fa08bc4ff9d3 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:46.998889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 639/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 28 09:01:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:47.005013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:47.113638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b17834-360f-461a-928e-4c46de2b0546 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:47.148123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:47.148743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:47.149021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:47.149320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] PATH_INFO: `/v3/services` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:47.150147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:47.150451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:47.151835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66015f14-c686-439d-a91f-9b7a211449a1 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:47.156234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 653/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 28 09:01:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:49.163005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b073-2864-4bcf-a75b-9713430bc2e8 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:49.163692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b073-2864-4bcf-a75b-9713430bc2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:49.163692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b073-2864-4bcf-a75b-9713430bc2e8 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:49.164928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 640/1293] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:49.174066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7b9b-347f-4797-8f77-b8d4c9e77ef7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:49.174622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7b9b-347f-4797-8f77-b8d4c9e77ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:49.174622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7b9b-347f-4797-8f77-b8d4c9e77ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:49.193686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0daf7b9b-347f-4797-8f77-b8d4c9e77ef7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:49.194903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0daf7b9b-347f-4797-8f77-b8d4c9e77ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:49.224542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:49.262812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 654/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:49.287792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96b073-2864-4bcf-a75b-9713430bc2e8 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:49.321723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:49.398417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96b073-2864-4bcf-a75b-9713430bc2e8 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:49.433551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:49.434401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:49.434683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:49.434961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:49.435657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:49.437022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41528c2c-c919-475f-8435-6bf0857021a0 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:49.451884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 641/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Jul 28 09:01:49 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:01:51.524089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479e97-44e4-4d8c-ac06-699980f9ce29 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:51.524676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479e97-44e4-4d8c-ac06-699980f9ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:51.524676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479e97-44e4-4d8c-ac06-699980f9ce29 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:51.526025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 655/1296] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:51.531417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:51.531900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:51.531994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:51.550098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:51.551294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:51.637044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 642/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:51.659385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83479e97-44e4-4d8c-ac06-699980f9ce29 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:51.761343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83479e97-44e4-4d8c-ac06-699980f9ce29 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:51.801873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:51.802516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:51.802793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:51.803064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:51.803695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:51.805072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:51.815119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:51.816359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:51.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 656/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 09:01:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:51.822623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:51.844884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:51.937619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ed97-d243-450f-8e46-b3702da61761 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:51.971452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:51.972080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:51.972363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:51.972640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:51.973349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:51.973656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:51.975067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:51.980326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 643/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 28 09:01:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:51.989253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:52.021822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:52.094619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a28a8-f898-47b1-8109-6c7afff8f4a8 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:52.129378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:52.130039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:52.130324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:52.130602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:52.131370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] RBAC: Adding query filter params (interface=public, service_id=c314d9c8f5f3452cb7aea203c63518c7, region_id=RegionOne) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:52.131684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:52.133067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0ee30-210f-4856-b470-d7b352c37145 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:52.138390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 657/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Jul 28 09:01:51 2022] GET /identity/v3/endpoints?service_id=c314d9c8f5f3452cb7aea203c63518c7&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:52.144185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:52.238267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:52.271486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a4c-9503-42fa-bc69-fcac9546fc08 None admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:52.306744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:52.307370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:52.307651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:52.307924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:52.308580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:52.310542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769986e-2f10-4aa9-a801-2d0c96a4c2aa None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:52.319063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 644/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Jul 28 09:01:52 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:01:54.324630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272fa32-1242-40e1-b7ee-2b7479dd7509 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:54.325192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272fa32-1242-40e1-b7ee-2b7479dd7509 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:54.325192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272fa32-1242-40e1-b7ee-2b7479dd7509 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:54.327243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 658/1302] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:01:54.332987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:54.333468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:54.333560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:54.351691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:01:54.352900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:01:54.421590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 645/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:01:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 09:01:54.444629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272fa32-1242-40e1-b7ee-2b7479dd7509 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:54.541929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:54.549057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272fa32-1242-40e1-b7ee-2b7479dd7509 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:54.593493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:54.594116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:54.594470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:54.594744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:54.595526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:54.597055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 09:01:54.608595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 09:01:54.609737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application with Lock( Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self._enter() Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application return self.creator() Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application created_value = creator( Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: ERROR keystone.server.flask.application Jul 28 09:01:54.611150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 659/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Jul 28 09:01:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 09:01:54.616440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:54.698030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:01:54.721209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee1db-1ccb-4d69-a026-122d7830a535 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:54.757030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:54.757693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:54.758009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:54.758292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] PATH_INFO: `/v3/services` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:54.758980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:01:54.759287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:54.760799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1dd67-d95b-4b16-93d4-070c1959c955 None admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:54.766771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 646/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Jul 28 09:01:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:01:54.772871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:01:54.874707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f10af-49c7-4df9-9a3d-5a0b23788c22 None admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:01:54.910375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:01:54.910998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:01:54.911279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:01:54.911552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:01:54.912273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:01:54.913655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09b0ce-7228-4d7a-a83d-84f1dd4c1a54 None admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:01:54.936387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 660/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:01:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:02:04.763236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=30 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:04.931448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:02:59.897977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306165d-8755-4b57-affd-1ff011b3465e None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:59.898455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306165d-8755-4b57-affd-1ff011b3465e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:59.898680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306165d-8755-4b57-affd-1ff011b3465e None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:59.905662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1515-443e-45e2-8603-8776c67d4007 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:02:59.906101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1515-443e-45e2-8603-8776c67d4007 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:02:59.906320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1515-443e-45e2-8603-8776c67d4007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:02:59.909289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 647/1307] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:02:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:00.062358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b6ca1515-443e-45e2-8603-8776c67d4007 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:00.064640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b6ca1515-443e-45e2-8603-8776c67d4007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:00.225046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 661/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:03:00.277439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:00.277797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:00.278238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:00.279844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 648/1309] 10.222.0.48 () {54 vars in 965 bytes} [Thu Jul 28 09:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:00.286048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2ba3-584f-4097-a94f-ce16973579fb None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:00.286338 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2ba3-584f-4097-a94f-ce16973579fb None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:00.286615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f2ba3-584f-4097-a94f-ce16973579fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:00.367946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b92f2ba3-584f-4097-a94f-ce16973579fb None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:00.369212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b92f2ba3-584f-4097-a94f-ce16973579fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:00.391320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:00.451936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 662/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Jul 28 09:03:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:03:00.460700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:00.462738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:00.576874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57e659-addf-456e-839f-3fbf450727f3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:00.653729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:00.654295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:00.654572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:00.654842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:00.657487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:00.735555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:00.798271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:00.799940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c43fee-bc3d-48bd-a6ce-a8aa876120c7 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:00.822543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 649/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:03.139267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ccf-cc1d-41d7-abce-c5e9e1e33c67 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:03.139788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ccf-cc1d-41d7-abce-c5e9e1e33c67 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:03.139868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ccf-cc1d-41d7-abce-c5e9e1e33c67 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:03.141320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 663/1312] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:03.146624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a320-583a-40b9-951b-39a8ab34add1 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:03.147100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a320-583a-40b9-951b-39a8ab34add1 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:03.147193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a320-583a-40b9-951b-39a8ab34add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:03.223186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-60c6a320-583a-40b9-951b-39a8ab34add1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:03.224307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-60c6a320-583a-40b9-951b-39a8ab34add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:03.334825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 650/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:03.389648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:03.389961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:03.390239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:03.391915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 664/1314] 10.222.0.48 () {54 vars in 965 bytes} [Thu Jul 28 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:03.398442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca492-7189-436c-8e69-71ce7144bf74 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:03.398763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca492-7189-436c-8e69-71ce7144bf74 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:03.399020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca492-7189-436c-8e69-71ce7144bf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:03.413085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:03.502706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-227ca492-7189-436c-8e69-71ce7144bf74 None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:03.503866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-227ca492-7189-436c-8e69-71ce7144bf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:03.592805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 651/1315] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Jul 28 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:03.602502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:03.604103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:03.725216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b510f-32d7-4f76-a4ac-2cb20cecae72 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:03.806693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:03.812244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:03.813489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:03.814055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:03.814574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:03.817867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:03.963880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:03.965277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e86676-d8f8-41ed-9adf-576b5f1c44ae service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:03.987833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 665/1316] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:06.588382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:06.589288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:06.589288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:06.590381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 652/1317] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:03:06.596786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc312be-82c6-4385-ba36-7752f8b6db50 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:06.597577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc312be-82c6-4385-ba36-7752f8b6db50 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:06.597577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc312be-82c6-4385-ba36-7752f8b6db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:06.662019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-abc312be-82c6-4385-ba36-7752f8b6db50 None None] MFA Rules not processed for user `e9282f610393452ab15e259a75209eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:06.663254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-abc312be-82c6-4385-ba36-7752f8b6db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:06.718929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:06.771579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 666/1318] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:06.811609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:06.813169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:06.920563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519746c-e8f1-4542-ac25-7aa3c9af888b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:06.923582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:06.994718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:06.995343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:06.996227 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:06.996506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:06.998174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:07.106297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:07.107744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b3160-8e60-41c1-80fe-31f7d55514a2 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:07.126807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 653/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:09.678992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:09.679577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:09.679577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:09.680930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 667/1320] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:09.687073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd240-f93b-4f90-a9d0-576b2525a552 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:09.687510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd240-f93b-4f90-a9d0-576b2525a552 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:09.687624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd240-f93b-4f90-a9d0-576b2525a552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:09.728998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:09.756260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-eefcd240-f93b-4f90-a9d0-576b2525a552 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:09.757516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eefcd240-f93b-4f90-a9d0-576b2525a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:09.852872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 654/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:09.892524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:09.894092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:09.997346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eae6e-98e1-45a0-9e5f-65519f409d73 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:10.068735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:10.069356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:10.069638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:10.069940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:10.071589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:10.086925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:10.201702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:10.203200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62b339-d722-46b8-a86b-3d9802812c31 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:10.231589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 668/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:12.979159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:12.979716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:12.979716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:12.981119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 655/1323] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:12.986330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b6e2-82c0-424e-b401-52a82930f079 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:12.986808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b6e2-82c0-424e-b401-52a82930f079 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:12.986916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b6e2-82c0-424e-b401-52a82930f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:13.056556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8937b6e2-82c0-424e-b401-52a82930f079 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:13.057753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8937b6e2-82c0-424e-b401-52a82930f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:13.116860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:13.158311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 669/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:13.195789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:13.197432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:13.314236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf1b77-e4f6-441b-a77f-d0c04e65a3e0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:13.327890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:13.391784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:13.392414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:13.392695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:13.392966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:13.394625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:13.511852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:13.513593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d9d75-aca8-40d7-a0d2-6ba73946bb36 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:13.532069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 656/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:15.646492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:15.647009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:15.647009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:15.648301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 670/1326] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:15.654241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaca85-b2d4-4df0-aadb-b4fb117f986c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:15.654726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaca85-b2d4-4df0-aadb-b4fb117f986c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:15.654932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaca85-b2d4-4df0-aadb-b4fb117f986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:15.706605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:15.726200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4bbaca85-b2d4-4df0-aadb-b4fb117f986c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:15.727407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4bbaca85-b2d4-4df0-aadb-b4fb117f986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:15.826609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 657/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:15.863472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:15.865137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:15.990746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8111-57c9-41ff-8bb7-703233097a9b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:16.066788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:16.067439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:16.067705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:16.067977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:16.069708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:16.088167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:16.182186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:16.184112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88facb46-8e3a-4907-b053-e7a7af4994d7 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:16.203336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 671/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:18.288431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:18.288964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:18.288964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:18.290314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 658/1329] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:18.295561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d96a-49dd-4229-9d16-a1d0f19e9581 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:18.295989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d96a-49dd-4229-9d16-a1d0f19e9581 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:18.296187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d96a-49dd-4229-9d16-a1d0f19e9581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:18.372072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2634d96a-49dd-4229-9d16-a1d0f19e9581 None None] MFA Rules not processed for user `e9282f610393452ab15e259a75209eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:18.373284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2634d96a-49dd-4229-9d16-a1d0f19e9581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:18.442050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:18.475855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 672/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:03:18.513908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:18.515447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:18.616983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa2d2-d31a-44f4-a11d-07de6986d337 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:18.636792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:18.690463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:18.691101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:18.691383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:18.691727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:18.693436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:18.811827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:18.813280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaf417-5676-46a5-ac36-6de391e467fd service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:18.832125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 659/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:21.322736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:21.323321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:21.323321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:21.325433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 673/1332] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:21.330678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232486c-fcc1-42e5-96f3-7d2494c80414 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:21.331139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232486c-fcc1-42e5-96f3-7d2494c80414 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:21.331235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232486c-fcc1-42e5-96f3-7d2494c80414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:21.393694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:21.396165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8232486c-fcc1-42e5-96f3-7d2494c80414 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:21.398090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8232486c-fcc1-42e5-96f3-7d2494c80414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:21.492825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 660/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:21.529607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:21.531209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:21.637007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143a199-341d-421f-a434-f76cb88c5b67 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:21.713946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:21.714688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:21.714972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:21.715251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:21.716910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:21.751797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:21.851253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:21.852708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3387fe1-077d-409f-8b0b-8294a24c24b2 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:21.871721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 674/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:23.931787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:23.932288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:23.932413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:23.934365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 661/1335] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:03:23.940268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7288b-fb93-42a3-8c06-339c8b122672 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:23.940751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7288b-fb93-42a3-8c06-339c8b122672 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:23.940848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7288b-fb93-42a3-8c06-339c8b122672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:24.010803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5da7288b-fb93-42a3-8c06-339c8b122672 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:24.011977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5da7288b-fb93-42a3-8c06-339c8b122672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:24.087324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:24.110268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 675/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:24.150182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:24.151841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:24.262278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc4f58-ba5c-4461-a26b-674ed2946304 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:24.290516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:24.335350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:24.335968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:24.336248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:24.336519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:24.338169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:24.445770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:24.447231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373999d-066f-46aa-92ed-0e78f1edc273 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:24.465786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 662/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:26.975190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:26.975717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:26.975834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:26.977121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 676/1338] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:26.982922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946817-1200-4ecd-9fd5-e1ab945ef81a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:26.983321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946817-1200-4ecd-9fd5-e1ab945ef81a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:26.983501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946817-1200-4ecd-9fd5-e1ab945ef81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:27.048365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-37946817-1200-4ecd-9fd5-e1ab945ef81a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:27.049491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-37946817-1200-4ecd-9fd5-e1ab945ef81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:27.057633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:27.144357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 663/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:27.180270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:27.181869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:27.283120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ec89-c9ed-418d-8c1d-abb88d22341a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:27.354191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:27.354756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:27.355036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:27.355308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:27.356941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:27.394042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:27.473610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:27.475093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ab7ef-45ee-446a-8c56-a04d03410a14 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:27.493113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 677/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:31.600310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:31.600822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:31.600822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:31.602196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 664/1341] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:31.607533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b023f9-2065-4d55-ae8a-fb6f3c496679 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:31.607950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b023f9-2065-4d55-ae8a-fb6f3c496679 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:31.608113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b023f9-2065-4d55-ae8a-fb6f3c496679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:31.682246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-41b023f9-2065-4d55-ae8a-fb6f3c496679 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:31.683478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-41b023f9-2065-4d55-ae8a-fb6f3c496679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:31.778454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:31.790486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 678/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:31.830867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:31.832515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:31.944643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb128bab-37e5-4b82-b5a1-1e931a212bdc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:31.994097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:32.032091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:32.032719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:32.033000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:32.033273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:32.034962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:32.156546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:32.158060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28051154-f62b-4e7e-9401-6ea8366ddcdb service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:32.178324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 665/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:03:34.846957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:34.847717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:34.847717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:34.848893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 679/1344] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:34.856334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b17f1-cfe8-4774-99d4-ec794435a0b7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:34.856924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b17f1-cfe8-4774-99d4-ec794435a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:34.857326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b17f1-cfe8-4774-99d4-ec794435a0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:34.946566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-566b17f1-cfe8-4774-99d4-ec794435a0b7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:34.947843 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-566b17f1-cfe8-4774-99d4-ec794435a0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:34.968718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:35.056393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 666/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:35.094394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:35.095966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:35.205441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776971f-6d30-415e-ac12-8a6b6da3eed5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:35.289559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:35.290129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:35.290404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:35.290674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:35.292812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:35.337243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:35.408006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:35.409592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faa472-e91f-4b9f-8a01-9a712c789184 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:35.431656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 680/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:39.571563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:39.572232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:39.572232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:39.573455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 667/1347] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:39.579361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b237-07e6-4a56-bfae-8af36d52bddb None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:39.579808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b237-07e6-4a56-bfae-8af36d52bddb None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:39.579900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b237-07e6-4a56-bfae-8af36d52bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:39.650435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-aec1b237-07e6-4a56-bfae-8af36d52bddb None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:39.652346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aec1b237-07e6-4a56-bfae-8af36d52bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:39.800245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:39.804100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 681/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:39.843253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:39.844803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:39.982663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831ad1-3fa1-4eea-9e30-5b1b90e47884 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:40.049789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:40.082251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:40.083006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:40.083373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:40.083742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:40.085511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:40.339413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:40.340754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c40f5-785d-4ffd-b749-83e58ca101ac service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:40.359611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 668/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:03:45.387267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e235f7-b701-44a6-8ffa-6727f2bbab82 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:45.387787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e235f7-b701-44a6-8ffa-6727f2bbab82 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:45.388204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e235f7-b701-44a6-8ffa-6727f2bbab82 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:45.389749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 682/1350] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:45.396991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dd9f5-04f3-48bf-a9e0-854088bafb72 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:45.397469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dd9f5-04f3-48bf-a9e0-854088bafb72 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:45.397546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dd9f5-04f3-48bf-a9e0-854088bafb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:45.539363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ff4dd9f5-04f3-48bf-a9e0-854088bafb72 None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:45.540485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ff4dd9f5-04f3-48bf-a9e0-854088bafb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:45.583391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:45.708619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 669/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:03:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:50.115100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402438-7fbe-4986-bd70-ef792c2d23c1 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:50.115663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402438-7fbe-4986-bd70-ef792c2d23c1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:50.115663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402438-7fbe-4986-bd70-ef792c2d23c1 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:50.117176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 683/1352] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:50.122619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77397-0db5-4a23-8684-47113541853a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:50.123503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77397-0db5-4a23-8684-47113541853a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:50.124098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77397-0db5-4a23-8684-47113541853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:50.209402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5ae77397-0db5-4a23-8684-47113541853a None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:50.211118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5ae77397-0db5-4a23-8684-47113541853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:50.292381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:50.331558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 670/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:03:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:51.696659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4b42-6e0e-4451-8536-05cf52ca2d92 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:51.697237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4b42-6e0e-4451-8536-05cf52ca2d92 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:51.697237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4b42-6e0e-4451-8536-05cf52ca2d92 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:51.699378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 684/1354] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:51.704398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010526-5ed2-4e65-8925-7696ff2740ee None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:51.704840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010526-5ed2-4e65-8925-7696ff2740ee None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:51.704962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010526-5ed2-4e65-8925-7696ff2740ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:51.780810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-18010526-5ed2-4e65-8925-7696ff2740ee None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:51.782043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-18010526-5ed2-4e65-8925-7696ff2740ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:51.888639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 671/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:03:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:55.185344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba829b9-1c7c-498b-9b10-c23ec1921781 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:55.186465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba829b9-1c7c-498b-9b10-c23ec1921781 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:55.186915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba829b9-1c7c-498b-9b10-c23ec1921781 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:55.192356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 685/1356] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:55.194282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dedf0-793f-4b2c-b969-9237f172c4ce None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:55.194737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dedf0-793f-4b2c-b969-9237f172c4ce None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:55.195021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dedf0-793f-4b2c-b969-9237f172c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:55.250221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:55.292466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c83dedf0-793f-4b2c-b969-9237f172c4ce None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:55.293722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c83dedf0-793f-4b2c-b969-9237f172c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:55.396393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 672/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:55.449782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:55.450100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:55.450570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:55.451938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 686/1358] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 28 09:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:55.459938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fdc10-16a0-43f3-a026-36f61ead1637 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:55.460380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fdc10-16a0-43f3-a026-36f61ead1637 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:55.461080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fdc10-16a0-43f3-a026-36f61ead1637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:55.565049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c38fdc10-16a0-43f3-a026-36f61ead1637 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:55.566377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c38fdc10-16a0-43f3-a026-36f61ead1637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:55.618431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:55.693146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 673/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 28 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:03:55.701034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:55.702731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:55.860805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3f36-dc7c-42c7-aeea-304a9a7a5e27 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:55.956041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:56.001275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:56.002341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:56.002810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:56.003556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:56.005404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:56.113225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:56.115286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42aa46-430c-4e7d-9db8-9f7a21daac66 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:56.136828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 687/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:03:58.421910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:58.422415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:58.422415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:58.423885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 674/1361] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:03:58.428825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd54-f51a-4b5d-8490-40d26d3c068a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:58.429284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd54-f51a-4b5d-8490-40d26d3c068a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:58.429385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd54-f51a-4b5d-8490-40d26d3c068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:58.498650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-05eefd54-f51a-4b5d-8490-40d26d3c068a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:03:58.499855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-05eefd54-f51a-4b5d-8490-40d26d3c068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:58.539762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:58.618154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 688/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:03:58.665858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:03:58.668261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:58.857385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cb9d1-98e1-4be2-aa58-f2178ad6c635 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:03:58.872826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:03:58.985954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:03:58.986580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:03:58.986860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:03:58.987129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:03:58.988754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:03:59.080354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:03:59.081986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51680d1-28cb-460f-ac0d-1628694c8e08 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:03:59.100571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 675/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:01.455099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183e3a-da53-4311-aa16-a69a3ed0a358 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:01.456019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183e3a-da53-4311-aa16-a69a3ed0a358 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:01.456298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183e3a-da53-4311-aa16-a69a3ed0a358 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:01.457672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 689/1364] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:01.463442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5cb2-6d12-4827-a118-361a55f4ae68 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:01.464335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5cb2-6d12-4827-a118-361a55f4ae68 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:01.464607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5cb2-6d12-4827-a118-361a55f4ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:01.486061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:01.546956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3fde5cb2-6d12-4827-a118-361a55f4ae68 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:01.548177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3fde5cb2-6d12-4827-a118-361a55f4ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:01.672900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 676/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:01.717870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:01.718241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:01.718586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:01.720627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 690/1366] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 28 09:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:01.726006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769b5c-ab34-42f0-af3d-11b7c00cb78f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:01.726304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769b5c-ab34-42f0-af3d-11b7c00cb78f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:01.726581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769b5c-ab34-42f0-af3d-11b7c00cb78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:01.808507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6a769b5c-ab34-42f0-af3d-11b7c00cb78f None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:01.809723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6a769b5c-ab34-42f0-af3d-11b7c00cb78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:01.845793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:01.949432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 677/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 28 09:04:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:01.958348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:01.960063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:02.115475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c49671-5226-4d89-ba00-92ad046580a4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:02.138053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:02.241375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:02.241944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:02.242223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:02.242494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:02.244100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:02.339715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:02.341442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747028a-debd-48c4-90e5-96ff776ac8b4 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:02.360727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 691/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:03.310349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e539b8-f733-4cf8-bb51-329a357343bd None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:03.310906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e539b8-f733-4cf8-bb51-329a357343bd None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:03.310906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e539b8-f733-4cf8-bb51-329a357343bd None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:03.313895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 678/1369] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:03.317888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017902-849c-4799-9c42-a0f663749123 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:03.318302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017902-849c-4799-9c42-a0f663749123 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:03.318517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4017902-849c-4799-9c42-a0f663749123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:03.361794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:03.400404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c4017902-849c-4799-9c42-a0f663749123 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:03.402218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c4017902-849c-4799-9c42-a0f663749123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:03.508206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 692/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:03.853474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:03.853474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:03.853474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:03.855242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 679/1371] 10.222.0.48 () {54 vars in 964 bytes} [Thu Jul 28 09:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:03.859634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316e8-c194-4a9f-ab88-0f20bc3c2c07 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:03.859956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316e8-c194-4a9f-ab88-0f20bc3c2c07 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:03.860217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316e8-c194-4a9f-ab88-0f20bc3c2c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:03.947633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e99316e8-c194-4a9f-ab88-0f20bc3c2c07 None None] MFA Rules not processed for user `40d10d1a51524ba2967695566bb78e42`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:03.949385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e99316e8-c194-4a9f-ab88-0f20bc3c2c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:03.997765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:04.029225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 693/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Jul 28 09:04:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:04.036595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:04.038126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:04.148870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90c6d-4f0a-4c1b-83ec-cc233bb1161b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:04.191150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:04.230717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:04.231332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:04.231606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:04.231878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:04.233590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:04.353670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:04.355692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013d1a0-61d0-476f-9353-6c6dfd9d9dab service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:04.375510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 680/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:04.441520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:04.441984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:04.442523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:04.444702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 694/1374] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:04.450699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da204f-3ed8-4c0a-8011-2de290f360fb None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:04.451202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da204f-3ed8-4c0a-8011-2de290f360fb None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:04.451521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da204f-3ed8-4c0a-8011-2de290f360fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:04.517255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-97da204f-3ed8-4c0a-8011-2de290f360fb None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:04.518434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-97da204f-3ed8-4c0a-8011-2de290f360fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:04.540142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:04.622156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 681/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:04.662655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:04.664376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:04.824062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518fef-e8e5-4a5e-84cb-f5b28411eba8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:04.874111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:04.939629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:04.940258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:04.940538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:04.940811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:04.942522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:05.031253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:05.032994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877e436-e194-40d5-a954-4b27b6a8b071 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:05.051499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 695/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:07.072747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:07.073358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:07.073358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:07.074744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 682/1377] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:07.080167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8070-f4d2-4b9a-a7e9-27cf379f220a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:07.080556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8070-f4d2-4b9a-a7e9-27cf379f220a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:07.080735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8070-f4d2-4b9a-a7e9-27cf379f220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:07.146447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5e5b8070-f4d2-4b9a-a7e9-27cf379f220a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:07.147664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5e5b8070-f4d2-4b9a-a7e9-27cf379f220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:07.155211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:07.249455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 696/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:07.294943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:07.296592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:07.486942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33462259-a1f7-48a3-8d80-9b7b543fc6d8 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:07.490536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:07.606109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:07.606730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:07.607009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:07.607283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:07.608902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:07.709756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:07.711451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388eb1-e87c-405b-860d-93cf3d27ec89 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:07.730114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 683/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:09.804752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:09.805324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:09.805324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:09.806887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 697/1380] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:09.812200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0ce7c-c917-4eda-97f4-af279ed5e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:09.812649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0ce7c-c917-4eda-97f4-af279ed5e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:09.812738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0ce7c-c917-4eda-97f4-af279ed5e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:09.824831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:09.882269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f6f0ce7c-c917-4eda-97f4-af279ed5e2d3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:09.883382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f6f0ce7c-c917-4eda-97f4-af279ed5e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:10.009885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 684/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:10.051735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:10.053401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:10.201502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:10.214428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787eeff-6815-4d32-a423-be9aa3225b16 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:10.338744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:10.339862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:10.340267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:10.340643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:10.342448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:10.438501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:10.440197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba304d-8762-4b6b-9fad-2189ac0c30c3 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:10.451380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:10.461263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 698/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:12.615386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:12.616085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:12.616085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:12.617262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 685/1383] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:12.623976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fbed0-950f-46d1-acc2-bd529e29e219 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:12.624524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fbed0-950f-46d1-acc2-bd529e29e219 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:12.624661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fbed0-950f-46d1-acc2-bd529e29e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:12.706884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b35fbed0-950f-46d1-acc2-bd529e29e219 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:12.708405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b35fbed0-950f-46d1-acc2-bd529e29e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:12.819242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 699/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:12.861755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:12.863311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:12.962661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:13.042804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7105370-8d82-48cc-8e0a-6b1f6e9b9782 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:13.175344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:13.176493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:13.176863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:13.177264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:13.179628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:13.249534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:13.303131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:13.305395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae51e7-ca8b-48c8-9d44-d3fcca0943c7 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:13.332271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 686/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:04:15.785341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:15.785903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:15.785903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:15.787643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 700/1386] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:15.794385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ec99-6d10-423d-9c95-aaf5abc824ea None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:15.794842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ec99-6d10-423d-9c95-aaf5abc824ea None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:15.794947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ec99-6d10-423d-9c95-aaf5abc824ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:15.876712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5fd7ec99-6d10-423d-9c95-aaf5abc824ea None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:15.877991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5fd7ec99-6d10-423d-9c95-aaf5abc824ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:15.982935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 687/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:16.029451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:16.031583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:16.125584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:16.198340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d64dc-eaa5-4ddf-9ee2-4e3b3b6ce9bc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:16.324165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:16.325146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:16.325499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:16.325829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:16.327978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:16.377487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:16.443051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:16.445092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51ea44-5b93-4f9d-9e18-dcc5b1755fb8 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:16.475904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 701/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:18.516858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f8208-22c3-4226-a488-8820670065da None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:18.517469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f8208-22c3-4226-a488-8820670065da None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:18.517469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f8208-22c3-4226-a488-8820670065da None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:18.518894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 688/1389] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:18.524115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51668d-ff9f-4f39-9e4e-736ce5b18812 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:18.524573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51668d-ff9f-4f39-9e4e-736ce5b18812 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:18.524685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51668d-ff9f-4f39-9e4e-736ce5b18812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:18.590964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7c51668d-ff9f-4f39-9e4e-736ce5b18812 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:18.592178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7c51668d-ff9f-4f39-9e4e-736ce5b18812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:18.688602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 702/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:18.732876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f8208-22c3-4226-a488-8820670065da None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:18.734452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a04f8208-22c3-4226-a488-8820670065da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:18.765440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:18.889930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f8208-22c3-4226-a488-8820670065da None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:18.996533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:19.008078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:19.008704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:19.008986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:19.009260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:19.010979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:19.102094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:19.103681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1ea51-5137-4816-a912-ca2078120d4e service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:19.122277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 689/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:19.589365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed4f-7491-4318-a6ba-25e50f8b082f None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:19.589955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed4f-7491-4318-a6ba-25e50f8b082f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:19.589955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed4f-7491-4318-a6ba-25e50f8b082f None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:19.591332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 703/1392] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:19.596606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67992db9-d771-4c64-8e6c-068c45bf8b67 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:19.597019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67992db9-d771-4c64-8e6c-068c45bf8b67 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:19.597161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67992db9-d771-4c64-8e6c-068c45bf8b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:19.662277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-67992db9-d771-4c64-8e6c-068c45bf8b67 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:19.663466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-67992db9-d771-4c64-8e6c-068c45bf8b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:19.721496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:19.764243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 690/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:19.818841 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:19.819133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:19.819413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:19.820823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 704/1394] 10.222.0.48 () {54 vars in 966 bytes} [Thu Jul 28 09:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:19.826544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2333-dc17-420b-8139-2d14580c299f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:19.826833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2333-dc17-420b-8139-2d14580c299f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:19.827109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae2333-dc17-420b-8139-2d14580c299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:19.901166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-eaae2333-dc17-420b-8139-2d14580c299f None None] MFA Rules not processed for user `e4e8151c2541400cb76b8b19384f49d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:19.902318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eaae2333-dc17-420b-8139-2d14580c299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:19.981467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 691/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Thu Jul 28 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:19.988923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:19.990838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:20.005784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:20.095607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b5cc8-06f6-4ec4-828e-e56c17fde05c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:20.169884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:20.170424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:20.170704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:20.170985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:20.172594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:20.240332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:20.283979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:20.285418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed7990-7dd9-4c9a-aac7-29958dc47945 service glance] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:20.303587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 705/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:21.045524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4f99-014c-4d54-9b10-0b6bb94736aa None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.046058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4f99-014c-4d54-9b10-0b6bb94736aa None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.046058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4f99-014c-4d54-9b10-0b6bb94736aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.111333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bb7a4f99-014c-4d54-9b10-0b6bb94736aa None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:21.112542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bb7a4f99-014c-4d54-9b10-0b6bb94736aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.120769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:21.170276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaa046-976b-493f-980c-527e00dd688d None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.170850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaa046-976b-493f-980c-527e00dd688d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.170850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaa046-976b-493f-980c-527e00dd688d None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.172193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 706/1397] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:04:21.177259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.177547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.177826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.200237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 692/1398] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:21.251620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aab8a-cec1-4527-b15b-aad87b0d9904 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.252007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:21.252166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aab8a-cec1-4527-b15b-aad87b0d9904 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.252282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aab8a-cec1-4527-b15b-aad87b0d9904 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.253232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.253877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 693/1399] 10.222.0.48 () {54 vars in 943 bytes} [Thu Jul 28 09:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:04:21.264384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.265017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.265297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.349076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] MFA Rules not processed for user `59f92dd2b31f4a6dbb732ee67575ac1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:21.350308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.355577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 707/1400] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:04:21.400619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:21.402217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.427902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:21.435937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:21.458991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 694/1401] 10.222.0.48 () {58 vars in 1040 bytes} [Thu Jul 28 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:04:21.468383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:21.477950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc17d3a-c02b-4360-8da8-a697c618b250 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:21.520497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06563dc1-e4df-4dad-8241-0b637c61607a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:21.585400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:21.586062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.586340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.586618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.588323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.635485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:21.636036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:21.636317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:21.636588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:21.638312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:21.655792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:21.670721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:21.672309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db415ee-22bd-4f04-bf90-d20f3bf7df24 service swift] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:21.680372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 695/1402] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:04:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 09:04:21.733694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:21.735388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517332b8-78ff-4dbc-a344-674d9eb27314 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:21.755107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 708/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:04:23.828642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:23.829264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:23.829264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:23.831025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 696/1404] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:23.835830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804ba9-d6b8-4ea2-a6a1-aa375e250dec None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:23.836245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804ba9-d6b8-4ea2-a6a1-aa375e250dec None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:23.836407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804ba9-d6b8-4ea2-a6a1-aa375e250dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:23.912238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-15804ba9-d6b8-4ea2-a6a1-aa375e250dec None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:23.913524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-15804ba9-d6b8-4ea2-a6a1-aa375e250dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:24.022212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:24.029497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 709/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:24.078428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:24.079959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:24.090986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:24.244137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e0a95-dec1-4575-8195-6840bcf37796 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:24.337439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:24.371694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:24.372418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:24.372701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:24.372974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:24.374648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:24.471546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:24.473104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407417e-d2bf-47c8-8885-93b4ed8bacde service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:24.495428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 697/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:04:26.546063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:26.546624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:26.546624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:26.547982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 710/1407] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:26.553458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9da46-78fb-4b86-9630-0f649981dcbf None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:26.553995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9da46-78fb-4b86-9630-0f649981dcbf None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:26.554109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9da46-78fb-4b86-9630-0f649981dcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:26.627944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-37c9da46-78fb-4b86-9630-0f649981dcbf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:26.629505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-37c9da46-78fb-4b86-9630-0f649981dcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:26.682775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:26.743376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 698/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:26.805988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:26.807131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:27.008886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0718e-e521-45e5-895d-3dba71acd00b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:27.085929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:27.139954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:27.140586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:27.140864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:27.141262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:27.142928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:27.245185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:27.246986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d4968-bf6b-4859-93a1-4317bcf19d5a service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:27.271494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 711/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:36.741327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=23 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:37.267984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=64 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:44.016241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96279-f201-49f6-81a7-a2b8d368e6f1 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:44.016807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96279-f201-49f6-81a7-a2b8d368e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:44.016807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96279-f201-49f6-81a7-a2b8d368e6f1 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:44.018187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 699/1410] 10.222.0.48 () {54 vars in 918 bytes} [Thu Jul 28 09:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:44.023312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572b715-fde2-4320-80b1-78f53eaf33ab None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:44.023741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572b715-fde2-4320-80b1-78f53eaf33ab None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:44.023874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572b715-fde2-4320-80b1-78f53eaf33ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:44.097721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0572b715-fde2-4320-80b1-78f53eaf33ab None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:44.098934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0572b715-fde2-4320-80b1-78f53eaf33ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:44.203669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 712/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 28 09:04:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:44.218364 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e5a3c-0d12-40c4-b55a-169ccab01182 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:44.218691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e5a3c-0d12-40c4-b55a-169ccab01182 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:44.218963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e5a3c-0d12-40c4-b55a-169ccab01182 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:44.220332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 700/1412] 10.222.0.48 () {54 vars in 918 bytes} [Thu Jul 28 09:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:44.224747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56590-6994-426d-96a1-cfb05efb7c22 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:44.225030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56590-6994-426d-96a1-cfb05efb7c22 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:44.225302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56590-6994-426d-96a1-cfb05efb7c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:44.296209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:44.299437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-41e56590-6994-426d-96a1-cfb05efb7c22 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:44.301256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-41e56590-6994-426d-96a1-cfb05efb7c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:44.414660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 713/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 28 09:04:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:46.868015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:46.868533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:46.868533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:46.869902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:46.875011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905299-a2b4-4876-8991-d03e3fc57ba1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:46.875440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905299-a2b4-4876-8991-d03e3fc57ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:46.875564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905299-a2b4-4876-8991-d03e3fc57ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:46.941585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-df905299-a2b4-4876-8991-d03e3fc57ba1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:46.942833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-df905299-a2b4-4876-8991-d03e3fc57ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:46.996084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:47.038206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:47.080901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:47.082496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:47.228132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83765ef9-ed05-4d25-a4f9-510fd931841b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:47.305297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:47.346396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:47.347019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:47.347300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:47.347574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:47.349204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:47.435116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:47.436709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb9f82-b0ef-4dd6-bf17-91dbdf27c629 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:47.455156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 702/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:48.351950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961421d-e618-434f-8510-b827330e233c None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:48.352497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961421d-e618-434f-8510-b827330e233c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:48.352497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961421d-e618-434f-8510-b827330e233c None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:48.354433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:48.359350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366b67c-f788-4291-9a06-f5768e8ccddb None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:48.359786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366b67c-f788-4291-9a06-f5768e8ccddb None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:48.359890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366b67c-f788-4291-9a06-f5768e8ccddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:48.432686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0366b67c-f788-4291-9a06-f5768e8ccddb None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:48.433886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0366b67c-f788-4291-9a06-f5768e8ccddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:48.459691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:48.527193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:48.677609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961421d-e618-434f-8510-b827330e233c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:48.679111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9961421d-e618-434f-8510-b827330e233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:48.823114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961421d-e618-434f-8510-b827330e233c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:48.862028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:48.934545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:48.935172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:48.935448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:48.935719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:48.937610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:49.026123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:49.027746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec244b6-fe01-4eec-94fe-66302290efce service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:49.045167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 716/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:49.927512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:49.928011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:49.928115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:49.929422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:49.934976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639557e6-0cee-483a-8ec1-d7fa195bb76a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:49.935390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639557e6-0cee-483a-8ec1-d7fa195bb76a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:49.935536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639557e6-0cee-483a-8ec1-d7fa195bb76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:49.996292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:50.017920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-639557e6-0cee-483a-8ec1-d7fa195bb76a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:50.019060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-639557e6-0cee-483a-8ec1-d7fa195bb76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:50.115116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:50.268366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:50.269758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:50.413259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d5f7b-be2b-43a1-9a98-34b3b9f71497 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:50.425410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:50.524180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:50.524735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:50.525011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:50.525286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:50.527425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:50.614599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:50.616169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93114a4-f710-4e63-81c0-ac3ffd9bdd4d service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:50.633659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 705/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:51.490637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:51.491182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:51.491278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:51.492578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:51.498072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1d17-76e2-45a7-a34d-11460079a4c6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:51.498515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1d17-76e2-45a7-a34d-11460079a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:51.498623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1d17-76e2-45a7-a34d-11460079a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:51.518599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:51.575240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7d2b1d17-76e2-45a7-a34d-11460079a4c6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:51.576458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7d2b1d17-76e2-45a7-a34d-11460079a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:51.670945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:51.821269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:51.823096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:51.933028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:51.969479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53200e0b-f2b7-4c09-bdb8-2773a82cd24a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:52.205694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:52.206333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:52.206607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:52.206873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:52.208754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:52.282179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:52.296348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:52.297987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54724733-8485-468e-98b6-290e006c2a8b service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:52.316840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 719/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:53.172243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:53.172813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:53.173045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:53.174337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:53.179296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9579c0-77d7-4c6b-b99d-106f7e59993d None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:53.179742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9579c0-77d7-4c6b-b99d-106f7e59993d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:53.179858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9579c0-77d7-4c6b-b99d-106f7e59993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:53.252929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-da9579c0-77d7-4c6b-b99d-106f7e59993d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:53.254143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-da9579c0-77d7-4c6b-b99d-106f7e59993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:53.347942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:53.497194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:53.498686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:53.579084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:53.640671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6982b6-dcde-44fd-805f-8b12c9217c4c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:53.749571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:53.750153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:53.750424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:53.750769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:53.752385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:53.799666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:53.840570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:53.842191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef023b6-600f-45d1-a8bd-6ad2fe0d9175 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:53.860418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 708/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:54.659116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:54.659689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:54.659689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:54.661061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:54.665946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892750-89ff-499d-adec-7d115f610c10 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:54.666918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892750-89ff-499d-adec-7d115f610c10 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:54.667032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892750-89ff-499d-adec-7d115f610c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:54.742848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ef892750-89ff-499d-adec-7d115f610c10 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:54.744078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ef892750-89ff-499d-adec-7d115f610c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:54.839282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:54.990512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:54.992032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:55.040611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:55.141658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07449d-4e4a-4dd8-b080-acd12bfdecba None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:55.251771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:55.252306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:55.252582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:55.252853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:55.254547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:55.261315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:55.342344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:55.343930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff103e2f-edb3-42eb-ab89-f86d52c42ec9 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:55.361971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 722/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:56.168176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:56.168744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:56.168744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:56.170120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:56.174935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6d2-ad08-4d04-a7ca-108a9d73973f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:56.175374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6d2-ad08-4d04-a7ca-108a9d73973f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:56.175519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6d2-ad08-4d04-a7ca-108a9d73973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:56.251106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-57a4d6d2-ad08-4d04-a7ca-108a9d73973f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:56.252304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-57a4d6d2-ad08-4d04-a7ca-108a9d73973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:56.317190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:56.346659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:56.497954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:56.499394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:56.516485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:56.644293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac732f30-b1d3-4397-8675-2d391ddf222f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:56.728857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:56.754763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:56.755320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:56.755602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:56.755873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:56.757492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:56.847845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:56.849433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f8dda-0110-425a-8b35-196a32c59396 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:56.867507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 711/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:57.669250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:57.669879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:57.669879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:57.671210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:57.676239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4bb9c-d1e0-448d-978c-b44a0bf39231 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:57.676652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4bb9c-d1e0-448d-978c-b44a0bf39231 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:57.676795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4bb9c-d1e0-448d-978c-b44a0bf39231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:57.758403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4ed4bb9c-d1e0-448d-978c-b44a0bf39231 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:57.759623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4ed4bb9c-d1e0-448d-978c-b44a0bf39231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:57.800880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:57.855176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:58.005369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:58.007012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:58.168797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887fff-516e-407f-8d01-6843e27bb63d None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:58.230726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:58.306486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:58.307161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:58.307442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:58.307717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:58.313077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:58.416306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:58.418368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f05a98-a7e1-4c64-87d4-6498b19a80c6 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:58.436580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 725/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:04:59.246678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:59.247210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:59.247210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:59.248549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:04:59.253339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935f2f-a2c1-4e81-a760-cfdbb6af89d2 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:59.253789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935f2f-a2c1-4e81-a760-cfdbb6af89d2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:59.253927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935f2f-a2c1-4e81-a760-cfdbb6af89d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:59.330439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-33935f2f-a2c1-4e81-a760-cfdbb6af89d2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:04:59.331637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-33935f2f-a2c1-4e81-a760-cfdbb6af89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:59.333619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:59.427446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:04:59.577019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:04:59.578874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:59.723624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6776a4b-c8e0-4ede-84aa-14a98d4eb02c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:04:59.749894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:04:59.834066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:04:59.834612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:04:59.834887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:04:59.835157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:04:59.836758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:04:59.924127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:04:59.925745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8c0c-62cf-422b-a582-c627ee0dd69b service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:04:59.944856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 714/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:01.841859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:01.842378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:01.842474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:01.843773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:01.848965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39829-d866-41ab-9afa-4d41c9530bcb None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:01.849395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39829-d866-41ab-9afa-4d41c9530bcb None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:01.849511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39829-d866-41ab-9afa-4d41c9530bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:01.890136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:01.915207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9cd39829-d866-41ab-9afa-4d41c9530bcb None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:01.917001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9cd39829-d866-41ab-9afa-4d41c9530bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:02.016812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:02.054499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:02.056032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:02.163977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1315b9d-2569-4e42-9bf5-78e77e530048 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:02.196831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:02.236738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:02.237356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:02.237635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:02.237942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:02.239563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:02.344776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:02.346246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92ca8-892e-4dec-9aca-3ecdbbdf985c service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:02.364146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 728/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:04.489255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:04.489892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:04.489892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:04.491194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:04.497002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9144-1e41-44e3-8938-f5a8d439d215 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:04.497422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9144-1e41-44e3-8938-f5a8d439d215 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:04.497556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9144-1e41-44e3-8938-f5a8d439d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:04.561249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-250e9144-1e41-44e3-8938-f5a8d439d215 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:04.562478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-250e9144-1e41-44e3-8938-f5a8d439d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:04.575119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:04.661609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:04.702972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:04.704521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:04.804601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39d320-df3b-42f9-a443-edc5135edc42 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:04.816551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:04.876957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:04.877572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:04.877873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:04.878152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:04.879763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:04.989623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:04.991109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0d07e-1ff0-4131-99d5-f4a92463e329 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:05.008849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 717/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:07.318221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:07.318800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:07.318800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:07.320141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:07.326175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031d373-90a8-44e4-bffb-8c39b7a69ed9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:07.326606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031d373-90a8-44e4-bffb-8c39b7a69ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:07.326721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031d373-90a8-44e4-bffb-8c39b7a69ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:07.373868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:07.391992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4031d373-90a8-44e4-bffb-8c39b7a69ed9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:07.393224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4031d373-90a8-44e4-bffb-8c39b7a69ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:07.488373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:05:07.526951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:07.528506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:07.631800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ce88e-3c60-4786-a5bc-fce73ac588dc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:07.672756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:07.708076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:07.708689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:07.708966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:07.709240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:07.710888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:07.816483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:07.817937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672114f6-101c-4062-94ae-7f89cf445a2a service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:07.835814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 731/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:09.929880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:09.930390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:09.930497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:09.931813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:09.937283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf409b-9203-446b-8f7f-64bbaa1c9973 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:09.937698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf409b-9203-446b-8f7f-64bbaa1c9973 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:09.938137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf409b-9203-446b-8f7f-64bbaa1c9973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:10.003199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-07bf409b-9203-446b-8f7f-64bbaa1c9973 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:10.004409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-07bf409b-9203-446b-8f7f-64bbaa1c9973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:10.024868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:10.099144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:10.138504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:10.140097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:10.245076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6989d-f8f1-4ea4-a0ed-be4ac601fd31 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:10.265553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:10.321335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:10.321978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:10.322269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:10.323095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:10.324726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:10.431848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:10.433887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac49971-4bd4-49e4-8f26-85d0b97b559f service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:10.452154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 720/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:12.509861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:12.510448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:12.510448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:12.512422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:12.517603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b67b0-4fd6-4560-ad12-a89a1f1a530a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:12.518098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b67b0-4fd6-4560-ad12-a89a1f1a530a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:12.518197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b67b0-4fd6-4560-ad12-a89a1f1a530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:12.584021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:12.587081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-133b67b0-4fd6-4560-ad12-a89a1f1a530a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:12.588458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-133b67b0-4fd6-4560-ad12-a89a1f1a530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:12.684283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:12.723602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:12.725163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:12.829467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65266f52-176a-46c3-b3ba-dab9c56b89d5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:12.881542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:12.905666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:12.906314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:12.906601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:12.906871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:12.908575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:13.016310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:13.017744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803e5a-4a9a-4c57-9aec-e25dc6d11bb2 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:13.035909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 734/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:15.132380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:15.132940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:15.132940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:15.134316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:15.139462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e0336-a692-4778-b1a2-096db60e7187 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:15.139912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e0336-a692-4778-b1a2-096db60e7187 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:15.140021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e0336-a692-4778-b1a2-096db60e7187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:15.205608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ac9e0336-a692-4778-b1a2-096db60e7187 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:15.206765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ac9e0336-a692-4778-b1a2-096db60e7187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:15.234914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:15.302350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:15.344660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:15.346340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:15.474464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60fcf2-87ca-4ba5-8293-c3c684545991 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:15.502664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:15.547723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:15.548339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:15.548620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:15.548890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:15.550534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:15.657288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:15.658741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec672-0cd8-41b8-bada-ed742fab5a98 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:15.676641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 723/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:17.726843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:17.727390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:17.727390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:17.729356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:17.734636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec941d2-af21-4ee3-9b8e-a8b4555fefa7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:17.735049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec941d2-af21-4ee3-9b8e-a8b4555fefa7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:17.735195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec941d2-af21-4ee3-9b8e-a8b4555fefa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:17.801933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5ec941d2-af21-4ee3-9b8e-a8b4555fefa7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:17.803140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5ec941d2-af21-4ee3-9b8e-a8b4555fefa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:17.811191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:17.898326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:17.935756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:17.937316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:18.039556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7b1d2-fab4-426e-bd3c-e016ab9c22e5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:18.093480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:18.114432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:18.115059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:18.115341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:18.115616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:18.117261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:18.227389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:18.228842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0120e-a1fd-4a7f-822a-cfe7a4a64280 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:18.255224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 737/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:20.290440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94000d92-b48d-4392-83a3-4de711261366 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:20.291011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94000d92-b48d-4392-83a3-4de711261366 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:20.291011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94000d92-b48d-4392-83a3-4de711261366 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:20.292359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:20.298145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c77b5-350f-4516-aab4-a470e82c5fde None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:20.298527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c77b5-350f-4516-aab4-a470e82c5fde None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:20.298701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c77b5-350f-4516-aab4-a470e82c5fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:20.364012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b15c77b5-350f-4516-aab4-a470e82c5fde None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:20.365220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b15c77b5-350f-4516-aab4-a470e82c5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:20.402891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:20.461228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:20.505483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94000d92-b48d-4392-83a3-4de711261366 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:20.507100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-94000d92-b48d-4392-83a3-4de711261366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:20.610192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94000d92-b48d-4392-83a3-4de711261366 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:20.646060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:20.681612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:20.682268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:20.682548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:20.682827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:20.684444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:20.789396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:20.790855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa89a53-798c-4c3b-a14a-f1d0cf96880d service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:20.811829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 726/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:22.811571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:22.812151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:22.812151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:22.813760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:22.819030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babc2f7-2d45-4c73-b455-2df5facd52f0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:22.819423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babc2f7-2d45-4c73-b455-2df5facd52f0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:22.819580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babc2f7-2d45-4c73-b455-2df5facd52f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:22.883940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9babc2f7-2d45-4c73-b455-2df5facd52f0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:22.885401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9babc2f7-2d45-4c73-b455-2df5facd52f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:22.903994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:22.983077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:23.020243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:23.021832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:23.122606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588370b-ec4c-4754-9526-d56752531a5f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:23.186049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:23.193207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:23.193820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:23.194124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:23.194391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:23.195983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:23.299892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:23.301316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae0294-eb91-42e5-83dc-395b09cf5ea9 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:23.319494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 740/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:25.316195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:25.316747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:25.316747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:25.318197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:25.323505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66b41-8439-48f9-81db-aafc3bc61740 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:25.324487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66b41-8439-48f9-81db-aafc3bc61740 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:25.324593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66b41-8439-48f9-81db-aafc3bc61740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:25.388330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-10f66b41-8439-48f9-81db-aafc3bc61740 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:25.389521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-10f66b41-8439-48f9-81db-aafc3bc61740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:25.438693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:25.486498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:25.525211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:25.526800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:25.629029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffca09-ad36-4022-8dd8-f84cf9e3a283 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:25.675273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:25.701499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:25.702135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:25.702414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:25.702684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:25.704385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:25.809889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:25.811325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feea76-ac21-4574-8075-6eaf675d9cfe service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:25.830283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 729/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:35.484697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=22 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:35.827965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=66 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:45.295933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c884-0fd4-4399-b53b-f7ba306a7932 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:45.296550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c884-0fd4-4399-b53b-f7ba306a7932 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:45.296550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c884-0fd4-4399-b53b-f7ba306a7932 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:45.297961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 742/1471] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:45.303277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d217-ddf6-4b3f-a234-34edd92f3be4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:45.303719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d217-ddf6-4b3f-a234-34edd92f3be4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:45.303834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d217-ddf6-4b3f-a234-34edd92f3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:45.371015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-5b21d217-ddf6-4b3f-a234-34edd92f3be4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:45.372216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5b21d217-ddf6-4b3f-a234-34edd92f3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:45.465238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 730/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:45.547850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:45.548189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:45.548468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:45.549856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 743/1473] 10.222.0.48 () {54 vars in 966 bytes} [Thu Jul 28 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:45.554639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966bbc-3d75-44b5-8c07-e83689bbbd0e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:45.554924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966bbc-3d75-44b5-8c07-e83689bbbd0e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:45.555199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966bbc-3d75-44b5-8c07-e83689bbbd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:45.631574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-47966bbc-3d75-44b5-8c07-e83689bbbd0e None None] MFA Rules not processed for user `e4e8151c2541400cb76b8b19384f49d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:45.632763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-47966bbc-3d75-44b5-8c07-e83689bbbd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:45.652598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:45.707137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 731/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Thu Jul 28 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:45.714959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:45.716470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:45.819571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24136c-5479-4d24-b81f-3ca97974405b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:45.890322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:45.890941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:45.891220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:45.891491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:45.893097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:45.950886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:46.005174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:46.006646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2984b7-6649-403b-b9ed-6fd5133ed194 service glance] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:46.025238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 744/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:05:48.108705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:48.109231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:48.109231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:48.110619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 732/1476] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:48.115849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ffc81-0bb8-4745-857c-a444d716950f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:48.116239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ffc81-0bb8-4745-857c-a444d716950f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:48.116391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ffc81-0bb8-4745-857c-a444d716950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:48.179997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b66ffc81-0bb8-4745-857c-a444d716950f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:48.181181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b66ffc81-0bb8-4745-857c-a444d716950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:48.277191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 745/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:48.299666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:48.301175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:48.423218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bade2f-f089-441e-8f32-e93de301ed5e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:48.480573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:48.526238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:48.526847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:48.527123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:48.527390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:48.528070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:48.529859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:05:48.529859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:05:48.529859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:05:48.531014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc3b13-2954-4ada-8065-d1bea6d17cbd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:48.548988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 733/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Jul 28 09:05:48 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:05:50.577692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:50.578289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:50.578289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:50.579645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 746/1479] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:50.585446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d117de-d3c5-49f6-b847-64d493c8e463 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:50.585892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d117de-d3c5-49f6-b847-64d493c8e463 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:50.586031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d117de-d3c5-49f6-b847-64d493c8e463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:50.649437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-46d117de-d3c5-49f6-b847-64d493c8e463 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:50.650722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-46d117de-d3c5-49f6-b847-64d493c8e463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:50.745178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 734/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:50.787532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:50.789143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:50.799375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:50.931556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f96f3-303e-4056-a4ce-89362deb15c3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:51.009113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:51.042403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:51.043023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:51.043303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:51.043575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:51.045203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:51.132375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:51.134002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33d8ce-9f52-4a01-8441-951d5b1ac2e2 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:51.152401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 747/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:53.324619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:53.325172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:53.325172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:53.326577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 735/1482] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:53.331723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1c27-60c8-4c21-aa88-7c4c6d431b81 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:53.332167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1c27-60c8-4c21-aa88-7c4c6d431b81 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:53.332265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1c27-60c8-4c21-aa88-7c4c6d431b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:53.396765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-56cb1c27-60c8-4c21-aa88-7c4c6d431b81 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:53.397997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-56cb1c27-60c8-4c21-aa88-7c4c6d431b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:53.443374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:53.494082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 748/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:53.530553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:53.532093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:53.555711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:53.633244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c57b9-f700-4459-9439-67634fbf8d96 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:53.704029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:53.704642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:53.704923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:53.705196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:53.706833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:53.778211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:53.812486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:53.814209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75cfdf-3274-47a5-a0e5-aa2ccc35c368 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:53.834211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 736/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:55.878497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:55.879023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:55.879136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:55.880419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 749/1485] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:55.885554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52607-3ef8-4c96-98d4-9cfa5e9933bd None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:55.885994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52607-3ef8-4c96-98d4-9cfa5e9933bd None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:55.886741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52607-3ef8-4c96-98d4-9cfa5e9933bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:55.955874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-42e52607-3ef8-4c96-98d4-9cfa5e9933bd None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:55.957083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-42e52607-3ef8-4c96-98d4-9cfa5e9933bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:56.057209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 737/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:56.095710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:56.097272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:56.200488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363804ba-daeb-4a86-970a-b9981055aca7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:56.269333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:56.271268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:56.272026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:56.272526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:56.272983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:56.274820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:56.380539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:56.381932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39b305-6567-4f12-b4ca-c81f7c9225a0 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:56.400153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 750/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:05:58.586298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:58.586800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:58.586898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:58.588256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 738/1488] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:05:58.593406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f8027-35c1-4b1a-9d91-eb2c6093a802 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:58.593864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f8027-35c1-4b1a-9d91-eb2c6093a802 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:58.594009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f8027-35c1-4b1a-9d91-eb2c6093a802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:58.657425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-309f8027-35c1-4b1a-9d91-eb2c6093a802 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:05:58.658664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-309f8027-35c1-4b1a-9d91-eb2c6093a802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:58.710296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:58.754478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 751/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:05:58.795262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:05:58.796807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:58.831704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:58.898431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b9b51-6259-4183-ba8a-95b6c628114b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:05:58.970564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:05:58.971117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:05:58.971397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:05:58.971671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:05:58.973282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:05:59.053283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:05:59.083108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:05:59.084610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9783d-1cdd-4307-838e-c562e4edb0ed service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:05:59.102442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 739/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:06:01.461085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:06:01.461703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:06:01.461703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:06:01.463125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 752/1491] 10.222.0.48 () {54 vars in 926 bytes} [Thu Jul 28 09:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:06:01.468662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bde5e-4f43-449c-a42e-83c14c283fb8 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:06:01.469136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bde5e-4f43-449c-a42e-83c14c283fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:06:01.469280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bde5e-4f43-449c-a42e-83c14c283fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:06:01.534653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-fe9bde5e-4f43-449c-a42e-83c14c283fb8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:06:01.535850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fe9bde5e-4f43-449c-a42e-83c14c283fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:06:01.630199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 740/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Thu Jul 28 09:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:06:01.669964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:06:01.671527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:06:01.771981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0257019-61ac-42ce-939a-4972604d01ee None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:06:01.849606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:06:01.850321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:06:01.850597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:06:01.850862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:06:01.852466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:06:01.861848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:06:01.961936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:06:01.963390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd7e13-e835-44b3-92ff-0e22e4c00b3a service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:06:01.981845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 753/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:06:11.628370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=82 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:06:11.979477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=50 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:28.476148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:28.476749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:28.476749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:28.558563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:28.559806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:28.652614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 741/1494] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:07:28.660603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:28.661090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:28.661206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:28.736718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:28.737864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:28.830614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 754/1495] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:07:28.835201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:28.836470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:28.848372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc3c2d-b278-49cc-850b-651773d0f643 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:28.914835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:28.950772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:28.951388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:28.951664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:28.951933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:28.952616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:07:28.952995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:28.954038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:28.954038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:07:28.954038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:07:28.954917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:28.966314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 742/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:07:28.974636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:28.984423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b75af-892d-4a72-b697-042e9fd59f6a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:29.048906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:29.082751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:29.083382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.083661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.083933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.084604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:29.086356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:29.086356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:07:29.086356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:07:29.087551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:29.146040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 755/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:07:28 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:29.150265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:29.152419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:29.184348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc394674-f635-462f-a88f-576e87dd9209 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:29.282350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:29.282970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.283246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.283514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.284180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:29.285229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:29.285229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:07:29.285229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:07:29.286463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:29.328543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 743/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:07:29 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:07:29.332939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:29.334491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:29.395152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7ae5-9899-4caf-87a4-5bd2829b5586 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:29.476909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:29.487890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:29.488514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.488792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.489064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.489758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:29.490798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:29.490798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:07:29.490798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:07:29.491685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:29.504263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 756/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:07:29.508326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:29.510089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:29.521977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699c03-73fd-4474-9ea2-a43d9a2e52f7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:29.545850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:29.617071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:29.617607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.617904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.618173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.618854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:29.620404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:29.620404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:07:29.620404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:07:29.621279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfff03-65e3-4a8f-9ca1-31ad9cbb3cae admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:29.633314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 744/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:07:29.638161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:29.639746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:29.672443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8b5a9-1d7b-46cf-8897-05f55b1013e0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:29.765508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:29.766215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.766498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.766777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] PATH_INFO: `/v3/projects/2adf68e573ea4fd8b6c57a4c5d1b136e/users/b1f340df762b4d43b6bb03e6a7808a2b/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.787934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adf68e573ea4fd8b6c57a4c5d1b136e, user_id=b1f340df762b4d43b6bb03e6a7808a2b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:29.788975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:29.788975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:07:29.788975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:07:29.790967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd45f1d-ff8c-4da1-9193-cefab3ef1f85 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:29.814650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 757/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:07:29 2022] PUT /identity/v3/projects/2adf68e573ea4fd8b6c57a4c5d1b136e/users/b1f340df762b4d43b6bb03e6a7808a2b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:07:29.827480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d944965-9b36-4138-a41e-f401cf5125d6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.827768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d944965-9b36-4138-a41e-f401cf5125d6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.828045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d944965-9b36-4138-a41e-f401cf5125d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.884559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7d944965-9b36-4138-a41e-f401cf5125d6 None None] MFA Rules not processed for user `b1f340df762b4d43b6bb03e6a7808a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:29.885773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7d944965-9b36-4138-a41e-f401cf5125d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:29.963542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 745/1502] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:07:29.976708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.976999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.977306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:29.978709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 758/1503] 10.222.0.48 () {54 vars in 964 bytes} [Thu Jul 28 09:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:29.984022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efa69-ffae-49b2-8e7b-8d68d0305e1d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:29.984309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efa69-ffae-49b2-8e7b-8d68d0305e1d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:29.984613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efa69-ffae-49b2-8e7b-8d68d0305e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:30.014694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:30.059215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ab4efa69-ffae-49b2-8e7b-8d68d0305e1d None None] MFA Rules not processed for user `40d10d1a51524ba2967695566bb78e42`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:30.060405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ab4efa69-ffae-49b2-8e7b-8d68d0305e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:30.133581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 746/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Jul 28 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:07:30.141585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:30.143186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:30.195119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:30.250129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdba9a0-b2d2-40d5-b127-4b22d1726d46 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:30.321666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:30.322323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:30.322604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:30.323025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:30.324653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:30.422782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:30.424131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:30.427628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:30.442912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 759/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:07:30.488265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41d942-3f98-4e67-bc9e-d2c0313baf65 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:30.488555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41d942-3f98-4e67-bc9e-d2c0313baf65 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:30.488837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41d942-3f98-4e67-bc9e-d2c0313baf65 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:30.490162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 747/1506] 10.222.0.48 () {52 vars in 830 bytes} [Thu Jul 28 09:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:07:30.493728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:30.495017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:30.598533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba793-0a35-43db-b61a-75f2105a7f06 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:30.689391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:30.690023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:30.690266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:30.690544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] PATH_INFO: `/v3/users/b1f340df762b4d43b6bb03e6a7808a2b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:30.703247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f340df762b4d43b6bb03e6a7808a2b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:30.704271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:30.704271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:07:30.704271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:07:30.705482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:30.715397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:30.768280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f340df762b4d43b6bb03e6a7808a2b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:30.786929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f340df762b4d43b6bb03e6a7808a2b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:30.798763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f340df762b4d43b6bb03e6a7808a2b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:30.799240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f340df762b4d43b6bb03e6a7808a2b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:30.799768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f340df762b4d43b6bb03e6a7808a2b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:30.812977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 760/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:07:30 2022] DELETE /identity/v3/users/b1f340df762b4d43b6bb03e6a7808a2b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:07:30.817784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:30.818295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:30.818391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:30.875642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:07:30.876844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:30.950644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:30.964750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 748/1508] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:07:30.974817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:30.976048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:31.087562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719b6c-b349-4824-9e15-d5a353725e89 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:31.150662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:31.171614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:31.172245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:31.172523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:31.172795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:31.174456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:31.288498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:31.289970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3726ac7-4975-4e0a-a1e5-f946f1564d70 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:31.308108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 761/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:07:31.631939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:07:31.633254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:07:31.645122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a1283-a484-406a-b318-d564d30d929c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:07:31.743628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:07:31.744270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:07:31.744550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:07:31.744838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] PATH_INFO: `/v3/projects/2adf68e573ea4fd8b6c57a4c5d1b136e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:07:31.754543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adf68e573ea4fd8b6c57a4c5d1b136e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:07:31.755574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:07:31.755574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:07:31.755574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:07:31.756763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:07:31.812821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adf68e573ea4fd8b6c57a4c5d1b136e', 'request_id': 'req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:31.823474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adf68e573ea4fd8b6c57a4c5d1b136e', 'request_id': 'req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:31.836948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adf68e573ea4fd8b6c57a4c5d1b136e', 'request_id': 'req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:31.859971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] The token cache is being invalidate because project 2adf68e573ea4fd8b6c57a4c5d1b136e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:07:31.860262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:31.860698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-906e3b11-d0a2-4ccc-9399-cfc506db6e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:07:31.862165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 749/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:07:31 2022] DELETE /identity/v3/projects/2adf68e573ea4fd8b6c57a4c5d1b136e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:07:41.305344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=62 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:07:41.857985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:13.252964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.253528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.253528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.254878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:13.259765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54213468-2d65-4d56-a548-b4a995070785 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.260199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54213468-2d65-4d56-a548-b4a995070785 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.260306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54213468-2d65-4d56-a548-b4a995070785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.341085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-54213468-2d65-4d56-a548-b4a995070785 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:13.342315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-54213468-2d65-4d56-a548-b4a995070785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:13.441495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:13.595538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:13.597666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:13.712714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c252b00-af93-4eb5-80d0-95cda4b1c4d2 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:13.718149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:13.718768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:13.719055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:13.719328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:13.727798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:13.729378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b593694-5fa6-4a78-8653-da15ab3060a3 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:13.738610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 763/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:14.563130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed17f7-71e4-4bc2-bf2f-75426d141037 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.564272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed17f7-71e4-4bc2-bf2f-75426d141037 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.564272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed17f7-71e4-4bc2-bf2f-75426d141037 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.565599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:14.570569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944edb-f57c-4382-8c96-92032f68572c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.570967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944edb-f57c-4382-8c96-92032f68572c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.571125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944edb-f57c-4382-8c96-92032f68572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.590063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4a944edb-f57c-4382-8c96-92032f68572c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:14.591257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4a944edb-f57c-4382-8c96-92032f68572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:14.599016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:14.750109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed17f7-71e4-4bc2-bf2f-75426d141037 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:14.764144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed17f7-71e4-4bc2-bf2f-75426d141037 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:14.769458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:14.770090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:14.770368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:14.770643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:14.778553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:14.780135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46d93b-c02f-4d5b-8a31-32f6c406b6ae service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:14.789895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 752/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:15.323721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61f49-69cf-4f27-bc6d-ab75aff45714 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.324007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61f49-69cf-4f27-bc6d-ab75aff45714 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.324283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61f49-69cf-4f27-bc6d-ab75aff45714 None None] PATH_INFO: `/v3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.325585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 765/1517] 10.222.0.60 () {54 vars in 925 bytes} [Thu Jul 28 09:09:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:09:15.331861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e4f2-5261-479a-8264-44a61b764518 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:15.332150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e4f2-5261-479a-8264-44a61b764518 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:15.332453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5e4f2-5261-479a-8264-44a61b764518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:15.389409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-afa5e4f2-5261-479a-8264-44a61b764518 None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:15.390625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-afa5e4f2-5261-479a-8264-44a61b764518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:15.406663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:15.460345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 753/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jul 28 09:09:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:24.585307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=56 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:24.620130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4413-e2c8-4d4e-8c88-ab2215c813eb None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.620339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4413-e2c8-4d4e-8c88-ab2215c813eb None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.620639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4413-e2c8-4d4e-8c88-ab2215c813eb None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.622041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 766/1519] 10.222.0.48 () {54 vars in 961 bytes} [Thu Jul 28 09:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:24.627437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec73e-fe20-4b6d-ba6e-b683436863f4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.627923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec73e-fe20-4b6d-ba6e-b683436863f4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.628029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec73e-fe20-4b6d-ba6e-b683436863f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.647509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3f3ec73e-fe20-4b6d-ba6e-b683436863f4 None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:24.648691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3f3ec73e-fe20-4b6d-ba6e-b683436863f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:24.657386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 754/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Jul 28 09:09:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:24.664581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4413-e2c8-4d4e-8c88-ab2215c813eb None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:24.677674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4413-e2c8-4d4e-8c88-ab2215c813eb None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:24.682754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:24.683347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:24.683623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:24.683892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:24.692596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:24.694457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eed4f-5884-4907-b654-991928b3eacb service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:24.704619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 767/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:26.022522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49622a00-4b40-40a9-8697-5793ac4d4e01 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.023102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49622a00-4b40-40a9-8697-5793ac4d4e01 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.023102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49622a00-4b40-40a9-8697-5793ac4d4e01 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.024486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:26.029177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b841-d806-4b48-abb2-30cceb1f5b0a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.029571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b841-d806-4b48-abb2-30cceb1f5b0a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.029730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b841-d806-4b48-abb2-30cceb1f5b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.048614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0a21b841-d806-4b48-abb2-30cceb1f5b0a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:26.049804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0a21b841-d806-4b48-abb2-30cceb1f5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:26.057562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:26.211614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49622a00-4b40-40a9-8697-5793ac4d4e01 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:26.226059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49622a00-4b40-40a9-8697-5793ac4d4e01 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:26.231476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:26.232083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:26.232365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:26.232639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:26.240647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:26.242258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80f01-6246-456c-a5df-0e9a1908bfb9 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:26.252039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 756/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:27.061527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3337-3fe7-48c2-9d9f-a5d2f4fc99ca None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.062091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3337-3fe7-48c2-9d9f-a5d2f4fc99ca None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.062091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3337-3fe7-48c2-9d9f-a5d2f4fc99ca None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.063491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 09:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:27.068508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1928e-f8b7-42d4-ae88-66565d2916b9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.069183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1928e-f8b7-42d4-ae88-66565d2916b9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.069292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1928e-f8b7-42d4-ae88-66565d2916b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.088679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d0a1928e-f8b7-42d4-ae88-66565d2916b9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:27.089891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d0a1928e-f8b7-42d4-ae88-66565d2916b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:27.098877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 09:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:27.250222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3337-3fe7-48c2-9d9f-a5d2f4fc99ca None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:27.264452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3337-3fe7-48c2-9d9f-a5d2f4fc99ca None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:27.270633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:27.271244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:27.271522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:27.271796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:27.279964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:27.281539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f57bb-15d9-46a9-bf3a-d78ec2e54906 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:27.291347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 770/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:31.151895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.152461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.152461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.153865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:31.159049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e210f-9b80-42f5-9725-af71d83f5f3a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.159455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e210f-9b80-42f5-9725-af71d83f5f3a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.159625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e210f-9b80-42f5-9725-af71d83f5f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.178985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5a6e210f-9b80-42f5-9725-af71d83f5f3a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:31.180107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5a6e210f-9b80-42f5-9725-af71d83f5f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:31.187883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:31.260452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:31.262238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:31.345328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf94c5-c07a-44e7-a71f-a4dded208a6e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:31.348948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:31.349569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:31.349869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:31.350157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:31.359287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:31.361366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6d511-4871-4ef8-9986-6db6befa61f9 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:31.370778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 759/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:33.364069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e0d0-769c-45a0-8e4b-3f638ee0afc7 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.364643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e0d0-769c-45a0-8e4b-3f638ee0afc7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.364643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e0d0-769c-45a0-8e4b-3f638ee0afc7 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.366114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:33.371272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c00b-30e5-4591-ae77-4390d1803e66 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.371689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c00b-30e5-4591-ae77-4390d1803e66 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.371829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3c00b-30e5-4591-ae77-4390d1803e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.390980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b3f3c00b-30e5-4591-ae77-4390d1803e66 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:33.392714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b3f3c00b-30e5-4591-ae77-4390d1803e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:33.400611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:33.436627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e0d0-769c-45a0-8e4b-3f638ee0afc7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:33.448157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e0d0-769c-45a0-8e4b-3f638ee0afc7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:33.451808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:33.452418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:33.452699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:33.452973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:33.462601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:33.464059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8463aac-38bc-4788-8dd5-862ec348c703 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:33.473445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 773/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:37.297468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.298544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.298658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.300228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:37.305764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e824-416a-4c1c-9132-194440e12e22 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.306395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e824-416a-4c1c-9132-194440e12e22 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.306395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e824-416a-4c1c-9132-194440e12e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.325660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7959e824-416a-4c1c-9132-194440e12e22 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:37.326879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7959e824-416a-4c1c-9132-194440e12e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:37.334845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:37.637515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:37.639196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:37.703431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:37.720465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46705-9caf-4302-b60a-893d25d40990 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:37.724556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:37.725177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:37.725458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:37.725747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:37.734736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:37.736199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb72cdf-b328-451b-a3bc-848d304bbfd4 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:37.745551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 762/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:39.752638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.753184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.753184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.754536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 09:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:09:39.759738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038207-8975-4631-9488-3296437631f5 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:39.760167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038207-8975-4631-9488-3296437631f5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:39.760296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038207-8975-4631-9488-3296437631f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:39.779815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d6038207-8975-4631-9488-3296437631f5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:09:39.781007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d6038207-8975-4631-9488-3296437631f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:39.788823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 09:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:09:40.088789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:09:40.090490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:09:40.103162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0b27d-b643-44fc-ae11-94be6996f7f3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:09:40.106605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:09:40.107204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:09:40.107481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:09:40.107749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:09:40.116978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:09:40.118435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb0e96-516d-4854-9efe-6df7bc9241ce service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:09:40.128236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 776/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:09:49.775252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=11 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:09:50.123508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=33 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:11.005127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5105e-0310-4abc-b30b-2d20d10b27a7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.005895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5105e-0310-4abc-b30b-2d20d10b27a7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.005895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5105e-0310-4abc-b30b-2d20d10b27a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.034069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-30d5105e-0310-4abc-b30b-2d20d10b27a7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.035317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-30d5105e-0310-4abc-b30b-2d20d10b27a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.046828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 764/1540] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.056610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.057213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.057213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.086875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.087099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56776-e894-47e4-b136-73ec037c5328 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.087328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56776-e894-47e4-b136-73ec037c5328 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.087861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56776-e894-47e4-b136-73ec037c5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.088840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.102579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 777/1541] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.108250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.119536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-63c56776-e894-47e4-b136-73ec037c5328 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.120704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23002dc0-6ee6-4fa5-abf8-edad2417de71 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.122013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-63c56776-e894-47e4-b136-73ec037c5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.130604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.131311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.131588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.131885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.132596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.132905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.133959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:11.133959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:11:11.133959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:11:11.134886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.143277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 765/1542] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:11.153160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.153767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.154303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.154901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 778/1543] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:11:11.163112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.181300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f47c0-b9dd-495b-8dc6-51fcf0d60173 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.190724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.192114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.192796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.193092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.193772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.196058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.202958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.204457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.220806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 766/1544] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:11.226068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.241082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7e188-ddfe-48c6-9323-5a599e581d2a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.255173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.256216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.256726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.258182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.258182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.258740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.260675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.279454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 767/1545] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:11:11.288456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.288708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 779/1546] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:11.292764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.318180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff54aa-82d4-45f2-a90d-a38326c4e2d0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.417077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.417712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.418043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.418323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.418994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.420031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:11.420031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:11:11.420031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:11:11.421250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c9887-34f2-4da1-bc9f-e02904b6503e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.427317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95902-ca2c-4a0c-9d58-ad1061f16fac admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.432759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.433355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.433610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.433907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.434859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.435969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:11.435969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:11:11.435969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:11:11.436958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6115b9-07b5-436c-9a68-269eab55475d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.451458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 768/1547] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:11.454257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ff1b-03ce-4253-bd9e-48b061910685 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.454257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ff1b-03ce-4253-bd9e-48b061910685 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.454257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ff1b-03ce-4253-bd9e-48b061910685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.464366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 780/1548] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:11.468750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639ed48-d2f4-4d04-a4d1-a76691bcec21 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.469050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639ed48-d2f4-4d04-a4d1-a76691bcec21 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.469342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639ed48-d2f4-4d04-a4d1-a76691bcec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.474867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3d96ff1b-03ce-4253-bd9e-48b061910685 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.476161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3d96ff1b-03ce-4253-bd9e-48b061910685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.489430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f639ed48-d2f4-4d04-a4d1-a76691bcec21 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.490677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f639ed48-d2f4-4d04-a4d1-a76691bcec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.538477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 769/1549] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.539631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528604d0-4cf0-4625-9403-0ca004b9bd04 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.539924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528604d0-4cf0-4625-9403-0ca004b9bd04 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.540191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528604d0-4cf0-4625-9403-0ca004b9bd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.554255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 781/1550] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.555527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.555815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.556092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.567358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-528604d0-4cf0-4625-9403-0ca004b9bd04 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.568564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-528604d0-4cf0-4625-9403-0ca004b9bd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.575793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.576890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 770/1551] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.577010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.579045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.579341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.579653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.585444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 782/1552] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.586391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.598879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.600078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.602459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63844fa5-13ae-4ad9-b0ec-174829be5057 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.607046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.607658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.607938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.615762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 771/1553] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.615943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.616041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.616129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.616129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ec8e-a7d7-4e57-9601-a5cf620ecff0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.619185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3829c3-d487-43a7-a4b1-e1f19ed88d56 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.626101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.627028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.627350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.627808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.628669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.630404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba89086-e702-4fd6-a7bb-23afaa0ebafb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.637093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 783/1554] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:11.637093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3ee5-4b1e-42ab-938c-bdd42bd87289 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.637093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3ee5-4b1e-42ab-938c-bdd42bd87289 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.637093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3ee5-4b1e-42ab-938c-bdd42bd87289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.645805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 772/1555] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:11.645805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49019c1-ed69-448a-adc2-d4cbc1db6c5c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.645805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49019c1-ed69-448a-adc2-d4cbc1db6c5c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.645805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49019c1-ed69-448a-adc2-d4cbc1db6c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.656003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9e1f3ee5-4b1e-42ab-938c-bdd42bd87289 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.657211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9e1f3ee5-4b1e-42ab-938c-bdd42bd87289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.665266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c49019c1-ed69-448a-adc2-d4cbc1db6c5c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.665671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 784/1556] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.667022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b75b4-0d9b-4145-a9e9-a61f349de5ff None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.667308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b75b4-0d9b-4145-a9e9-a61f349de5ff None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.667419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c49019c1-ed69-448a-adc2-d4cbc1db6c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.667595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b75b4-0d9b-4145-a9e9-a61f349de5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.679238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 773/1557] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.680489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b8fd-36b9-4b99-b3d6-33a611706ed5 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.680775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b8fd-36b9-4b99-b3d6-33a611706ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.681053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b8fd-36b9-4b99-b3d6-33a611706ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.688459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a96b75b4-0d9b-4145-a9e9-a61f349de5ff None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.689591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a96b75b4-0d9b-4145-a9e9-a61f349de5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.697822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 785/1558] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.699106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.699393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.699671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.701159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8596b8fd-36b9-4b99-b3d6-33a611706ed5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.702689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8596b8fd-36b9-4b99-b3d6-33a611706ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.707325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:11.711093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 774/1559] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.712276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.712596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.712973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.724076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.725415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.732307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.733649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.737618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 786/1560] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.737618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.742857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 775/1561] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.743775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.745008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763f0d1-031e-410c-a9f8-308bf84d8467 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.749648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.750299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.750581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.750850 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.751549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.753140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554defb0-5145-4990-bcfa-0265cc2d0207 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.753790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0b11b-2d68-4f84-9176-b82b84b54906 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.758441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.759071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.759353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.759630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.760319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.761941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.763472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 787/1562] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:11.764378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:11.764641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.764931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.765208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.774931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 776/1563] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:11.775788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.784735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:11.785992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d393a71-1174-46ca-adf8-41cddb4479dd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.786569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:11.790731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.791331 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.791611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.791886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.792604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.792907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.794531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.798409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 788/1564] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:11.798409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.805756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2af418-fbad-4e1d-a458-b109bee80be5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.809709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 777/1565] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.810674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.817036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.817661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.817970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.818245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.818970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.819328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.821229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.826118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36b08-0746-4467-b5e8-57908d1f6319 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.830792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.831391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.831696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.831964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.832888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.833212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.833332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 789/1566] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.834118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.835620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.843939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa5cb9-acd3-4fd2-ac2e-7cd553cbcec8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.848538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 778/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.849532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.850190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.850510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.850789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.851490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.851810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.853403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.856376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.866494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77df49-7054-4a82-994f-60031a672f43 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.873647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 790/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.873647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.873909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.873909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.873909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.873909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.874288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.874288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.875566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.877166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447993c-c145-4251-976e-ede7a64ffc44 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.882271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.883355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.883639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.883917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.884613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.886256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.887556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 779/1569] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.888442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.897136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 791/1570] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:11.898015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.898262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522726d-c12d-4965-9bdd-7a1e75c432bb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.902974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.903661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.903944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.904216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] PATH_INFO: `/v3/projects/c957cfe3391147daab9fc1e59b414a3e/users/e0c1ed47784c46baa7ddc4abdfc54bbb/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.908115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9a025-08b6-4ad4-88d3-23fa3284129f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.913327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:11.913969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:11.914257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:11.914538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:11.915234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:11.915552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.917146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.926359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c957cfe3391147daab9fc1e59b414a3e, user_id=e0c1ed47784c46baa7ddc4abdfc54bbb, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:11.927019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:11.927019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:11:11.927019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:11:11.928965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:11.932279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 792/1571] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:11.933006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:11.944241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f435-4715-4fc6-86e0-d866f4b5dd31 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:11.950801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 780/1572] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:11 2022] PUT /identity/v3/projects/c957cfe3391147daab9fc1e59b414a3e/users/e0c1ed47784c46baa7ddc4abdfc54bbb/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:11.951812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.004995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.005625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.005932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.006203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.006860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.008756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.014172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71023a-3ea4-4361-a20e-a9537f2471fd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.018853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.019502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.019784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.020058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.020714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.022658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.036673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 793/1573] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.037604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.051153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 781/1574] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:11 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.052025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.106162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0530ae-3358-4460-b976-521fd7ed9a34 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.111025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.111645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.111923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.112193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.112310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed89a69-7866-4175-9dfa-8d5b323a9a49 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.112854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.115282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.117379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.118091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.118304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.118578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.119230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.121133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.135983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:12.145858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 794/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/projects => generated 353 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.146702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.149902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 782/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.150874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.157333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:12.209475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde65b-f853-46bd-9f42-868af7826745 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.214720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e797-a7b2-4c0a-94ed-be6693de1516 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.215930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.216522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.216889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.217166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.218012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.219847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.219980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.220515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.220793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.221065 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] PATH_INFO: `/v3/projects/ba0c305435a042aabbb61816788d7c66/users/816bcf956bb442e39ff9feb7e606dab4/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.235996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0c305435a042aabbb61816788d7c66, user_id=816bcf956bb442e39ff9feb7e606dab4, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.239345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e5318-fc3a-4365-bb2d-8a925cbacb44 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.249287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 795/1577] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.250061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.253831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 783/1578] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:12 2022] PUT /identity/v3/projects/ba0c305435a042aabbb61816788d7c66/users/816bcf956bb442e39ff9feb7e606dab4/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:12.258546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805610f0-4837-4088-b32f-78213f054c4d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.258834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805610f0-4837-4088-b32f-78213f054c4d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.259113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805610f0-4837-4088-b32f-78213f054c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.277463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-805610f0-4837-4088-b32f-78213f054c4d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:12.278697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-805610f0-4837-4088-b32f-78213f054c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:12.310990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb281f-6aad-47ad-99cf-e7d7307d42a1 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.315686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.316367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.316646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.316916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.317568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.319535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.337015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 784/1579] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:12.338081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805610f0-4837-4088-b32f-78213f054c4d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.348966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805610f0-4837-4088-b32f-78213f054c4d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.354640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 796/1580] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.354640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.354868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.354868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.355160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.355444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.356137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.358101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.380968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 785/1581] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.382062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.391294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91406673-f733-428f-96b5-2733e593b799 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.417151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81fe56-17a0-4bb3-8650-04bc41ebb9de admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.421965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.422595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.422879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.423156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.423822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.425811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.442037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:12.447759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.451552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.451552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.451787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 797/1582] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.451903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.452007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.452106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.452106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.460022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880f90-b2b7-4928-89e8-2687e76c8ad2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.464778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.465383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.465669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.465978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.466669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.468587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b0174-a650-45ad-9603-5c76deff62f6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.469678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:12.475797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 786/1583] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 277 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.476962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.487821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e73e8b-19cb-4349-9cab-e055cde0c743 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.489312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 798/1584] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.491524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.491919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.492525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.492977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.493574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.493877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.494154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.494803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.496696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.512229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:12.513435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:12.519020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 787/1585] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 294 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.519998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.524256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 799/1586] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:12.527946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.538195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09811e-9067-4c28-a53b-bfbd2b7819cd None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.543429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.544077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.544361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.544637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.545294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.547295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.572093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 800/1587] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:12 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:12.572093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.579939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffae98-0704-4514-af84-cd77d38459ab admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.584563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.584770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b0892-9539-47dc-8921-6a642aeaa582 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.585165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.585448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.585731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.586458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.588049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.588612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.589228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.589509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.589805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.598375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 801/1588] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.599299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.602603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.604063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.608829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3d650-ff1e-4ff4-a928-905717b717b5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.613941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.614236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 788/1589] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:12.614596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.614886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.615051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.615158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.615895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.617474 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.623899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b32b9-1a85-43e5-b968-fd04363ecf24 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.628789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 802/1590] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.631950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.631950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.631950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.631950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.632261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.632345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.632432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.641669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5080-ece7-4417-a002-b1bc6c3d909d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.646088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 789/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.646929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.647174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.648043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.648564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.649122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.650088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.651995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.657080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804194f-4395-41b7-a116-5c74fd710298 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.662369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.662981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.663389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.663669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.664370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.664799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 803/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.665639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.668981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.678374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2dc69-90e0-4a8a-9ee4-87db21147a12 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.684705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 790/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.684705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.685307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.686020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.686408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.686686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.697090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.698842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.706038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3572-2445-455a-91c2-fce4a66b68ed admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.710424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 804/1594] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:12.710988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.712455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.713325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.713617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.713933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.714751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.716725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.721537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78895fb-692a-46ab-b636-d9ce3b964164 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.727081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.727769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.728134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.728478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.729245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.730262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 791/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.731021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.731154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.745710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 805/1596] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.745933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b90fe7-4ca5-4451-bf05-f361aedac7ce admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.746037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.750773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.751760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.751997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.752813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.753523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.755619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.756306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94764f-8e99-4c93-b45d-542521787f70 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.760963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.761662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.761979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.762254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.762949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.764906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.768715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 792/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.770693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.778380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 806/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.779245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.787089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852577-26f5-4ca5-a914-9c9b5b6496a8 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.792680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acc486-c31b-4f8d-96e9-e2dc7ff5d8ce admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.794189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.795051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.795334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.795608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.796320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.797943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.798335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.798987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.799321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.799598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.800336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.801955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.819000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 793/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.819396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 807/1600] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.819488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.819566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.826250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e23d5-9dc2-4a6c-a2cf-8c510abfe8db admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.831491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.832222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa00272-137c-459d-b443-ca9176a5db00 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.832720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.832968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.833243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] PATH_INFO: `/v3/projects/d962b65c517449c7ad4a9ae343273c81/users/4fb1f69572964cffae3bd9aa19e2af11/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.839166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.839787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.840060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.840333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.841037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.842760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.854959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 794/1601] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:12.855785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.857131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d962b65c517449c7ad4a9ae343273c81, user_id=4fb1f69572964cffae3bd9aa19e2af11, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.860285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.865914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc5d27-bbc6-4b9f-b86b-eda656302cc9 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.871801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:12.872580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:12.872944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:12.873292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] PATH_INFO: `/v3/projects/17ee1d1560884281989fe65530189845/users/c5d075688fc044fcb5d89ddcd53fd71d/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:12.886060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 808/1602] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:12 2022] PUT /identity/v3/projects/d962b65c517449c7ad4a9ae343273c81/users/4fb1f69572964cffae3bd9aa19e2af11/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:12.886060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.897464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ee1d1560884281989fe65530189845, user_id=c5d075688fc044fcb5d89ddcd53fd71d, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:12.900470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:12.919160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 795/1603] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:12 2022] PUT /identity/v3/projects/17ee1d1560884281989fe65530189845/users/c5d075688fc044fcb5d89ddcd53fd71d/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:12.934984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:12.974199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b45f2-4cef-4196-b601-5897212489f1 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:12.995196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:13.009377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:13.016445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32122a67-e8c0-46a8-a2c8-1813ed7858cf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.025293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.026582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.027027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.027367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] PATH_INFO: `/v3/projects/6e862416d3ca4d019cd0dd62f439038c/users/a6f5e471a7ac4218a35ffe1840d91575/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.045407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e862416d3ca4d019cd0dd62f439038c, user_id=a6f5e471a7ac4218a35ffe1840d91575, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.048683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.053808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.054892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.055185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.055478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] PATH_INFO: `/v3/projects/7aaf9e3ae3b2410b915efb52c52a4ab9/users/236fae7fbbc242478c679e9459d2111b/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.065044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 796/1604] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:12 2022] PUT /identity/v3/projects/6e862416d3ca4d019cd0dd62f439038c/users/a6f5e471a7ac4218a35ffe1840d91575/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:13.066204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.075543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaf9e3ae3b2410b915efb52c52a4ab9, user_id=236fae7fbbc242478c679e9459d2111b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.079377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.099381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 809/1605] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:12 2022] PUT /identity/v3/projects/7aaf9e3ae3b2410b915efb52c52a4ab9/users/236fae7fbbc242478c679e9459d2111b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:13.100544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.156663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d842f35-a65c-4ff1-aaaa-7a721a9da0a0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.187861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059d06-cca5-49b9-a3f7-8d421c98ca09 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.198502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.199517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.199946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.200392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] PATH_INFO: `/v3/projects/b7b5e309462a425a92e550dd2c72ca4d/users/87b166b84ccd4300a1b862d375c137a5/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.220508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b5e309462a425a92e550dd2c72ca4d, user_id=87b166b84ccd4300a1b862d375c137a5, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.223514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703a57-3517-496e-9cd7-525948031d4b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.246362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.247930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.248321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.248781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] PATH_INFO: `/v3/projects/34cb634f5c154187a943a454a706f200/users/aff89a63c86f4e6f906988791f3c5258/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.258004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 810/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:13 2022] PUT /identity/v3/projects/b7b5e309462a425a92e550dd2c72ca4d/users/87b166b84ccd4300a1b862d375c137a5/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:13.259953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521543e-6b56-4365-a4b0-8f4ceff0c7a7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.260441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521543e-6b56-4365-a4b0-8f4ceff0c7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.260957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521543e-6b56-4365-a4b0-8f4ceff0c7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.272037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cb634f5c154187a943a454a706f200, user_id=aff89a63c86f4e6f906988791f3c5258, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.276394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1801b-fd90-44db-84bc-99dacf1c5198 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.294622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 797/1607] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:13 2022] PUT /identity/v3/projects/34cb634f5c154187a943a454a706f200/users/aff89a63c86f4e6f906988791f3c5258/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:13.296145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8087d-c913-4340-b6ef-de2ecf39faab None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.296530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8087d-c913-4340-b6ef-de2ecf39faab None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.296985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8087d-c913-4340-b6ef-de2ecf39faab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.301679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b521543e-6b56-4365-a4b0-8f4ceff0c7a7 None None] MFA Rules not processed for user `4fb1f69572964cffae3bd9aa19e2af11`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.303180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b521543e-6b56-4365-a4b0-8f4ceff0c7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.333413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-79a8087d-c913-4340-b6ef-de2ecf39faab None None] MFA Rules not processed for user `c5d075688fc044fcb5d89ddcd53fd71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.336246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-79a8087d-c913-4340-b6ef-de2ecf39faab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.406374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 811/1608] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.408487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22389a1-bba2-4fcb-99a8-600e3bf7e89d None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.410264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22389a1-bba2-4fcb-99a8-600e3bf7e89d None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.410465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:13.410617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22389a1-bba2-4fcb-99a8-600e3bf7e89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.413633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 798/1609] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.414996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.415520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.416079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.452408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c22389a1-bba2-4fcb-99a8-600e3bf7e89d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.453995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c22389a1-bba2-4fcb-99a8-600e3bf7e89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.466347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:13.498478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] MFA Rules not processed for user `236fae7fbbc242478c679e9459d2111b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.501341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.553647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 812/1610] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.555157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4363e-db8e-4236-9aab-73f97dbd3b8a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.555432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4363e-db8e-4236-9aab-73f97dbd3b8a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.555716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4363e-db8e-4236-9aab-73f97dbd3b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.586906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-94f4363e-db8e-4236-9aab-73f97dbd3b8a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.588714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-94f4363e-db8e-4236-9aab-73f97dbd3b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.599874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 813/1611] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.601242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f0e1-aeae-44fb-a85a-e8130226f02a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.601534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f0e1-aeae-44fb-a85a-e8130226f02a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.601814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f0e1-aeae-44fb-a85a-e8130226f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.619870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 799/1612] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.623350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.636592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-aad7f0e1-aeae-44fb-a85a-e8130226f02a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.637822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aad7f0e1-aeae-44fb-a85a-e8130226f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.644381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512880f5-ce6b-45cb-b68a-01181db19535 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.649949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 814/1613] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:13.650808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.651379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.651672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.651954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.652636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.653016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.653434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 815/1614] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 28 09:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:13.653562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.654347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.654666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.656212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.657375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.709468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 800/1615] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:13.710538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.740424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3369e-5fa2-411e-b3c0-d9736bd1b5e0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.800889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.802097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.802372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.802644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.804044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77af4d1-aeb6-48d0-98fb-6586c449a40d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.809557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.811258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.811887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.812347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.813189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.815541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.834981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:13.836442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b76b0-4b5d-4665-9a96-9d756d5b5bd2 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:13.857241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 801/1616] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/projects => generated 357 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:13.858305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:13.864404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 816/1617] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:13.866553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.866845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.867122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.941762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfde549-57bf-44e3-bb21-ebed373e4cbe admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:13.948331 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:13.949088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:13.949528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:13.949667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:13.950031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:13.950882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:13.958274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:14.046899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.049394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.066494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 802/1618] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:14.067490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:14.069112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.081828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c5a88-bed0-401f-874c-39e60f2623f1 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.086957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.087725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.088185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.091064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.116072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.119720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.122656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 817/1619] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 28 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:14.124087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.139677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7457a1-37a2-4807-93a9-fd148dbdf0d3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.147177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.147910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.148555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 803/1620] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:14.148667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.149040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.149730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.151113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.152044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.183520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289757c-d5d6-4d0a-bbbc-e5bfd3591b63 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.189724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 818/1621] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:14 2022] POST /identity/v3/users => generated 293 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:14.189724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.198791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.199449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.199736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.200166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.204429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.206480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.208685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d37e4-2aea-4b7c-b3e0-833f99603531 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.217443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.218420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.218844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.219246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.276331 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 804/1622] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:14 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:14.277222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.301854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb2f8-f670-4577-91be-93c54fba5c97 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.315362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.318086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.318371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.318652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.323110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.324954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.326886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.328788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.354985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 819/1623] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:14.355894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 805/1624] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:14.356487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.356804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.370759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c29f-13d5-4a66-b1de-7a51082fce6a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.382547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1eb79-ee74-4811-aded-02e80b44c6c6 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.387432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.388312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.388599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.388875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] PATH_INFO: `/v3/projects/b23776708ea84754b0b8d24c79ffffa6/users/90b3509a88ee49d88dec42183f2889b2/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.399013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.399674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.399954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.400230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.401089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.402872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.428128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23776708ea84754b0b8d24c79ffffa6, user_id=90b3509a88ee49d88dec42183f2889b2, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.431046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.432787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 820/1625] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:14.438157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.463516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a18730-cce1-4887-bd3c-08c1bd175c1c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.469810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 806/1626] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:14 2022] PUT /identity/v3/projects/b23776708ea84754b0b8d24c79ffffa6/users/90b3509a88ee49d88dec42183f2889b2/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:14.471322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.637655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9cc71-91e8-4fb9-b36e-3845b28835c1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.639087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.640266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.640563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.640839 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] PATH_INFO: `/v3/projects/8bba812cf9734deaba52757594a4e5aa/users/4694ab3e89b742e4b7741d45407f4297/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.652809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.653462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.653751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.654055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.654820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.659743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.678719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bba812cf9734deaba52757594a4e5aa, user_id=4694ab3e89b742e4b7741d45407f4297, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.681827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.685996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 807/1627] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:14.690272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.713686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 821/1628] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:14 2022] PUT /identity/v3/projects/8bba812cf9734deaba52757594a4e5aa/users/4694ab3e89b742e4b7741d45407f4297/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:14.717562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:14.717696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab3db-289e-4b35-80f6-9197859c888e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.784055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:14.812788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:14.847360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.848069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.849561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.850472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] PATH_INFO: `/v3/projects/b23776708ea84754b0b8d24c79ffffa6/users/90b3509a88ee49d88dec42183f2889b2/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.855594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23776708ea84754b0b8d24c79ffffa6, user_id=90b3509a88ee49d88dec42183f2889b2, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.860388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93a073-cdc4-4d30-90fb-e8c9f3353b3c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:14.875228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745125-fea2-4330-b05c-6cc560fe8f34 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:14.905145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 808/1629] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:14 2022] PUT /identity/v3/projects/b23776708ea84754b0b8d24c79ffffa6/users/90b3509a88ee49d88dec42183f2889b2/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:14.906900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81562656-7239-4276-84ee-080e0bd498a4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.907193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81562656-7239-4276-84ee-080e0bd498a4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.907536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81562656-7239-4276-84ee-080e0bd498a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.951978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-81562656-7239-4276-84ee-080e0bd498a4 None None] MFA Rules not processed for user `90b3509a88ee49d88dec42183f2889b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:14.953244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-81562656-7239-4276-84ee-080e0bd498a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:14.983969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:14.985106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:14.985388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:14.985795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:14.986604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:14.989481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e3451-c3c9-4d9e-8374-75851e15ae26 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.008200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 822/1630] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:11:15.008200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b3ef3-b830-4476-a7b4-6c9bf00656c5 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.008493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b3ef3-b830-4476-a7b4-6c9bf00656c5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.008787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b3ef3-b830-4476-a7b4-6c9bf00656c5 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.016115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 823/1631] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 28 09:11:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:15.017023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b3ef3-b830-4476-a7b4-6c9bf00656c5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:15.035041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b3ef3-b830-4476-a7b4-6c9bf00656c5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:15.041876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:15.042808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.043166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.043622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] PATH_INFO: `/v3/projects/8bba812cf9734deaba52757594a4e5aa/users/4694ab3e89b742e4b7741d45407f4297/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.046187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bba812cf9734deaba52757594a4e5aa, user_id=4694ab3e89b742e4b7741d45407f4297, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:15.049907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.052539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 809/1632] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:15.060686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3753c16-6877-4685-9a4d-a0d2a26707c0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.061075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3753c16-6877-4685-9a4d-a0d2a26707c0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.061608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3753c16-6877-4685-9a4d-a0d2a26707c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.069997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 824/1633] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:15 2022] PUT /identity/v3/projects/8bba812cf9734deaba52757594a4e5aa/users/4694ab3e89b742e4b7741d45407f4297/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:15.070859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:15.095063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d3753c16-6877-4685-9a4d-a0d2a26707c0 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:15.096673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d3753c16-6877-4685-9a4d-a0d2a26707c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:15.154894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45066e88-0ed3-4533-b380-a2a9b8bc12d4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:15.160971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:15.161974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.162848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.163208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.192629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 810/1634] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:15.195256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.195659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.196299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.235422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] MFA Rules not processed for user `4694ab3e89b742e4b7741d45407f4297`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:15.236840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:15.267548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:15.269026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.281661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 825/1635] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:15.282769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:15.284918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:15.293612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76bae3-dd31-4ac2-8d46-f0586a1dd1e5 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:15.299694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:15.301236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.302173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.302587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.315493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:15.318881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.328204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:15.336513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 826/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:15.349816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 811/1637] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:15.376534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:15.387675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a05f-4023-434e-89c8-726988b5b103 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:15.393792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:15.394848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.395046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.395373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.413402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:15.416070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.455635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 827/1638] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:15.673239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:15.693578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a31d-413c-4940-99fa-6d4d1cd19192 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:15.702251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:15.705760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:15.708096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:15.708613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:15.806830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:15.809064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c425-5473-4efc-8006-ce2e28b74106 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:15.822589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 812/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:16.314759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:16.328156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd8b-3efd-4048-a514-f5d7e33e59d1 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:16.337644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:16.338328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:16.338608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:16.338873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:16.589187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:16.594139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bd6fc-0180-484c-8abd-abce153e5a8f service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:16.657626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 828/1640] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:17.198157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df88e4-1148-4afa-8b77-c302ce57aac4 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:17.198771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df88e4-1148-4afa-8b77-c302ce57aac4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:17.198771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df88e4-1148-4afa-8b77-c302ce57aac4 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:17.200464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 813/1641] 10.222.0.48 () {54 vars in 962 bytes} [Thu Jul 28 09:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:17.218541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75bcfb-840d-4730-831c-9c048a7f981f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:17.219197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75bcfb-840d-4730-831c-9c048a7f981f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:17.219382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75bcfb-840d-4730-831c-9c048a7f981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:17.260602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-bb75bcfb-840d-4730-831c-9c048a7f981f None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:17.262035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bb75bcfb-840d-4730-831c-9c048a7f981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:17.282848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 829/1642] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Jul 28 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:17.290854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f712-eef8-4e82-b694-6b151dfe5706 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:17.291138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f712-eef8-4e82-b694-6b151dfe5706 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:17.291510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f712-eef8-4e82-b694-6b151dfe5706 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:17.293438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 814/1643] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 28 09:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:17.301107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0076-a284-4123-9063-328bfdf90de3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:17.301394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0076-a284-4123-9063-328bfdf90de3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:17.301667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0076-a284-4123-9063-328bfdf90de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:17.345170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-138f0076-a284-4123-9063-328bfdf90de3 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:17.346436 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-138f0076-a284-4123-9063-328bfdf90de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:17.372755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 830/1644] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:17.402278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f712-eef8-4e82-b694-6b151dfe5706 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:17.425791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f712-eef8-4e82-b694-6b151dfe5706 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:17.436058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:17.437634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:17.439238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:17.440405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:17.463763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:17.466154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90964ed-c50d-4942-b0c3-5342bb32697e service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:17.505954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 815/1645] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:23.006390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:23.006942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:23.007585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:23.075928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] MFA Rules not processed for user `816bcf956bb442e39ff9feb7e606dab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:23.082332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:23.283842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 831/1646] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:23.299365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c086e0e-54d7-4aef-8519-a6bada0df80f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:23.300241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c086e0e-54d7-4aef-8519-a6bada0df80f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:23.300701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c086e0e-54d7-4aef-8519-a6bada0df80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:23.341987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0c086e0e-54d7-4aef-8519-a6bada0df80f None None] MFA Rules not processed for user `816bcf956bb442e39ff9feb7e606dab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:23.344367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0c086e0e-54d7-4aef-8519-a6bada0df80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:23.367966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 816/1647] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:23.376540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:23.397978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758bca4-b9ba-4937-9d0f-431ac64c4ae6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:23.404535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:23.405337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:23.405720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:23.406130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:23.440299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:23.442123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed619bd-fc21-4f0a-9667-c8dfd5e7ac69 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:23.486354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 832/1648] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:24.875276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0775b1-3f0f-4789-8cc6-7fea3851a202 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:24.875826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0775b1-3f0f-4789-8cc6-7fea3851a202 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:24.876087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0775b1-3f0f-4789-8cc6-7fea3851a202 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:24.877701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 817/1649] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 28 09:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:24.883709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec189-574a-42a8-9d5d-f6f79f5ae2d3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:24.884374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec189-574a-42a8-9d5d-f6f79f5ae2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:24.884780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec189-574a-42a8-9d5d-f6f79f5ae2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:24.985768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4daec189-574a-42a8-9d5d-f6f79f5ae2d3 None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:24.989375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4daec189-574a-42a8-9d5d-f6f79f5ae2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:25.050187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:25.088100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 833/1650] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:25.103536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0775b1-3f0f-4789-8cc6-7fea3851a202 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:25.119103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0775b1-3f0f-4789-8cc6-7fea3851a202 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:25.128079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:25.131761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:25.132868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:25.133778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:25.150997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:25.157654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1293f2-827b-4354-86f5-1a5d10901316 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:25.175466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 818/1651] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:11:26.737864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9a27-39f4-4d7d-a1ea-6aa940bd6786 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:26.738322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9a27-39f4-4d7d-a1ea-6aa940bd6786 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:26.738322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9a27-39f4-4d7d-a1ea-6aa940bd6786 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:26.740136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 834/1652] 10.222.0.48 () {54 vars in 961 bytes} [Thu Jul 28 09:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:26.751072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c435-c699-4b8b-86cb-00b94aa5abac None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:26.751636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c435-c699-4b8b-86cb-00b94aa5abac None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:26.751636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c435-c699-4b8b-86cb-00b94aa5abac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:26.856996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-add9c435-c699-4b8b-86cb-00b94aa5abac None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:26.856996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-add9c435-c699-4b8b-86cb-00b94aa5abac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:27.095956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 819/1653] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Jul 28 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:27.109115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9a27-39f4-4d7d-a1ea-6aa940bd6786 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:27.144968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9a27-39f4-4d7d-a1ea-6aa940bd6786 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:27.156503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:27.157147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:27.157604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:27.158077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:27.163346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:27.207811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:27.210088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:27.252203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 835/1654] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:27.843572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167c74-59c5-4889-bd53-0c95d861f795 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:27.844144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167c74-59c5-4889-bd53-0c95d861f795 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:27.844144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167c74-59c5-4889-bd53-0c95d861f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:27.940445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f7167c74-59c5-4889-bd53-0c95d861f795 None None] MFA Rules not processed for user `e0c1ed47784c46baa7ddc4abdfc54bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:27.944245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f7167c74-59c5-4889-bd53-0c95d861f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:27.996005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:28.120158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 820/1655] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:28.147641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:28.165233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca177a-f1e3-4244-82e9-f4c841d9ddb2 service placement] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:28.184498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:28.185157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:28.185438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:28.185713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:28.211395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:28.214821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a98de9-9ade-4d07-8012-54b1af2d96eb service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:28.259548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 836/1656] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:28.804690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:28.804973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:28.805244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:28.864027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] MFA Rules not processed for user `aff89a63c86f4e6f906988791f3c5258`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:28.867345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:28.987576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 821/1657] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:29.000566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc43b-0718-4e83-9b72-e255ffda7f16 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:29.000857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc43b-0718-4e83-9b72-e255ffda7f16 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:29.001136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc43b-0718-4e83-9b72-e255ffda7f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:29.053793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5d9dc43b-0718-4e83-9b72-e255ffda7f16 None None] MFA Rules not processed for user `aff89a63c86f4e6f906988791f3c5258`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:29.055331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5d9dc43b-0718-4e83-9b72-e255ffda7f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:29.082414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 837/1658] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:29.092679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:29.118907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2f584-94e3-48f7-bdff-01a2523ea4ea None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:29.127877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:29.129063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:29.129502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:29.129789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:29.140200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:29.141914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda63e6-f80b-45e1-be1b-f1f3f7185225 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:29.181748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 822/1659] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:29.328163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:29.328750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:29.328930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:29.421202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] MFA Rules not processed for user `a6f5e471a7ac4218a35ffe1840d91575`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:29.422928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:29.602319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 838/1660] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:29.845814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d83f-e128-4cdc-9c94-64e0794bd52b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:29.849544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d83f-e128-4cdc-9c94-64e0794bd52b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:29.851048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d83f-e128-4cdc-9c94-64e0794bd52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:29.924391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4eb3d83f-e128-4cdc-9c94-64e0794bd52b None None] MFA Rules not processed for user `87b166b84ccd4300a1b862d375c137a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:29.933104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4eb3d83f-e128-4cdc-9c94-64e0794bd52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:29.968921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:30.002718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e0b30-f364-4e7b-a412-1d274262d4a1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:30.015379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:30.028461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:30.029804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:30.030250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:30.058659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:30.063506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:30.082642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 839/1661] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:30.106880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 823/1662] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:30.199287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:30.216763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c0d92-49d8-4f4a-907b-d34fd3f8ccbe service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:30.229868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:30.240186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:30.241660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:30.243173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:30.283224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:30.289697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a355f-07b4-447f-9d9b-2d5bb351c37e service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:30.325972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 840/1663] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:31.664545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325c07-1573-49d0-a829-9142af192aab None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:31.665021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325c07-1573-49d0-a829-9142af192aab None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:31.665129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325c07-1573-49d0-a829-9142af192aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:31.791961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:31.792831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:31.793395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:31.865833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:31.873350 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] MFA Rules not processed for user `aff89a63c86f4e6f906988791f3c5258`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:31.875062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:31.909250 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 841/1664] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:31.931725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7d325c07-1573-49d0-a829-9142af192aab None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:31.933174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7d325c07-1573-49d0-a829-9142af192aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:32.077818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 824/1665] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:32.386064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:32.387746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:32.604865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:32.670076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6becc1-31aa-4054-9a6d-0dddb29adbfc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:32.695421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:32.706019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:32.707450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:32.710083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:32.743321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:32.749417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88fbb5-d4a0-4850-9c4f-4da8db3e5148 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:32.798931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 842/1666] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:11:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:11:32.903622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d325c07-1573-49d0-a829-9142af192aab None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:33.146696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d325c07-1573-49d0-a829-9142af192aab None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:33.167979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:33.168929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:33.176198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:33.179700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] PATH_INFO: `/v3/users/c5d075688fc044fcb5d89ddcd53fd71d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:33.183302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d075688fc044fcb5d89ddcd53fd71d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:33.184543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: WARNING py.warnings [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:33.184543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: warnings.warn(msg) Jul 28 09:11:33.184543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: Jul 28 09:11:33.194526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:33.465800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d075688fc044fcb5d89ddcd53fd71d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:33.470131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d075688fc044fcb5d89ddcd53fd71d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:33.550714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d075688fc044fcb5d89ddcd53fd71d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:33.570760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d075688fc044fcb5d89ddcd53fd71d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:33.590858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d075688fc044fcb5d89ddcd53fd71d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:33.604867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 825/1667] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:32 2022] DELETE /identity/v3/users/c5d075688fc044fcb5d89ddcd53fd71d => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:33.608047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4306e8-72b9-4a8d-8cb2-3bb579ce7749 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:33.608259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4306e8-72b9-4a8d-8cb2-3bb579ce7749 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:33.608691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4306e8-72b9-4a8d-8cb2-3bb579ce7749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:33.675023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5d4306e8-72b9-4a8d-8cb2-3bb579ce7749 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:33.689746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5d4306e8-72b9-4a8d-8cb2-3bb579ce7749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:33.897310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 843/1668] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:34.062240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:34.063712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:34.136122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:34.265180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51282571-c873-44c2-a6c7-61141f6296ac admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:34.272940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:34.274107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:34.287075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:34.298197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:34.336767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:34.346085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64cd94-e4b3-4564-8538-b45f9e124c9d service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:34.397297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 826/1669] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:34.514170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075862a5-867e-4ac1-8367-94520e935557 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:34.514508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075862a5-867e-4ac1-8367-94520e935557 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:34.514807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075862a5-867e-4ac1-8367-94520e935557 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:34.517018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 844/1670] 10.222.0.48 () {54 vars in 912 bytes} [Thu Jul 28 09:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:34.531752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:34.532179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:34.532578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:34.599168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:34.619790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:34.641474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 827/1671] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:11:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:34.730675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075862a5-867e-4ac1-8367-94520e935557 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:34.733577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-075862a5-867e-4ac1-8367-94520e935557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:34.793226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075862a5-867e-4ac1-8367-94520e935557 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:34.808771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:34.813868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:34.817623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:34.819135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:34.862381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:34.874154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad712cc-2c2f-469a-ae43-eb9289234f64 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:34.958287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 845/1672] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:36.724688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:36.726773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:36.763186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389077cb-088c-4a3e-9b37-7bc2fb01329e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:36.803807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:36.804489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:36.804780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:36.805067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] PATH_INFO: `/v3/projects/17ee1d1560884281989fe65530189845` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:36.810545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ee1d1560884281989fe65530189845)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:36.812523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:36.970017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ee1d1560884281989fe65530189845', 'request_id': 'req-62753d8f-6ad7-449b-adca-3914a636e6ff'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:36.991907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ee1d1560884281989fe65530189845', 'request_id': 'req-62753d8f-6ad7-449b-adca-3914a636e6ff'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:37.015693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ee1d1560884281989fe65530189845', 'request_id': 'req-62753d8f-6ad7-449b-adca-3914a636e6ff'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:37.066305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] The token cache is being invalidate because project 17ee1d1560884281989fe65530189845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:37.073537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:37.075748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-62753d8f-6ad7-449b-adca-3914a636e6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:37.089025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:37.089025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:37.089025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:37.090136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 828/1673] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:36 2022] DELETE /identity/v3/projects/17ee1d1560884281989fe65530189845 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:37.226445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:37.228151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:37.624307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 846/1674] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:37.631720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:37.632271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:37.632774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:37.672743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:37.674167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:37.709408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:37.712015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 829/1675] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:37.755043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cffe4-f190-46e0-b39d-0d933f9a77ef None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:37.790834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:37.793421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:37.794148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:37.797015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:37.800726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:37.802510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:37.807990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df33496-b363-4bd7-9d0d-7bd1c4c98318 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:37.885108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:37.886302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 847/1676] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:37.913160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b9853-5607-469f-8935-68f7f53a3e9b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:37.948698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:37.950218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:37.953913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:37.955188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:37.960956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:37.967679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:38.078316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:38.078603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:38.078877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:38.100795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 830/1677] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:37 2022] POST /identity/v3/projects => generated 401 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:38.104425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:38.152140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] MFA Rules not processed for user `87b166b84ccd4300a1b862d375c137a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:38.154293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:38.405180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da510-8140-4787-8228-b8519971c8e1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:38.443286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:38.443629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:38.446014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:38.448204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:38.450820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:38.451609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:38.460730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:38.484205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 848/1678] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:38.516581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:38.551217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:38.552717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 831/1679] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:38 2022] POST /identity/v3/users => generated 301 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:38.702224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7f42-a113-4a66-9f4c-95ef2e39e8a5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:38.754816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:38.758641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:38.760271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:38.766310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:38.771409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:38.784301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:38.868188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:38.870630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 849/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:38.904090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680812b5-57e2-4f23-a7be-3fa860a7dfd4 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:38.928214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:38.931195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:38.937949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:38.938337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:38.941327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:38.953905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbc639-1184-49b3-a4e6-1353066b5392 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:39.129943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 832/1681] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:39.130636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:39.213411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9109c7-ee65-4819-b0cf-caba7d746c25 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:39.272432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:39.273988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:39.277561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:39.286594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] PATH_INFO: `/v3/projects/febb9e5ae0044129914cb4281642a68c/users/d0ad3ddc76664d94a82e270e44c74a8f/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:39.325671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598ad98-0041-4705-b8a9-6ccf24f93d89 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:39.326054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598ad98-0041-4705-b8a9-6ccf24f93d89 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:39.326500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598ad98-0041-4705-b8a9-6ccf24f93d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:39.396400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febb9e5ae0044129914cb4281642a68c, user_id=d0ad3ddc76664d94a82e270e44c74a8f, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:39.399486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525511-6cd1-4805-8cee-627043d33573 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:39.404432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0598ad98-0041-4705-b8a9-6ccf24f93d89 None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:39.407553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0598ad98-0041-4705-b8a9-6ccf24f93d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:39.443924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 850/1682] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:39 2022] PUT /identity/v3/projects/febb9e5ae0044129914cb4281642a68c/users/d0ad3ddc76664d94a82e270e44c74a8f/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:39.460525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:39.460840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:39.461110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:39.554179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] MFA Rules not processed for user `d0ad3ddc76664d94a82e270e44c74a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:39.555526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:39.591637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 833/1683] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jul 28 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:39.593087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:39.593543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:39.594010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:39.674603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] MFA Rules not processed for user `e0c1ed47784c46baa7ddc4abdfc54bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:39.676324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:39.815763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 851/1684] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:39.817365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:39.822652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:39.877934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 834/1685] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:39.878212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:40.055006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb400b8-6b13-4ea8-ab3a-dfab35fb108a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:40.077996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:40.078925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:40.079213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:40.079488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:40.134317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:40.135847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:40.140449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546656ef-77cd-4e88-bfb9-bd5801dd7843 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:40.166325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:40.167264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:40.167556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:40.167898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:40.168534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:40.170529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:40.199721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 852/1686] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:40.276402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 835/1687] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:39 2022] POST /identity/v3/projects => generated 401 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:40.281365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:40.298734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0d6b5-5b6d-46b8-ba5f-7e0b089b0fe9 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:40.554257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:40.593677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:40.597140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:40.602277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:40.606576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:40.611447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:40.619122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:40.668991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 853/1688] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:40 2022] POST /identity/v3/users => generated 307 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:40.674682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:40.704995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ed5a1-861a-46b4-b2f6-0d8e52168dfe admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:40.728231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:40.732125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:40.733739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:40.734246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:40.739729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:40.744813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:40.793621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 836/1689] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:40.795625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:40.819208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c10d-0e4f-401c-88f5-6f541807d6df admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:40.843028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:40.844866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:40.844866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:40.845091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] PATH_INFO: `/v3/projects/92e931a8708f48eb962baa3f93a7cd17/users/8079ddf9ea8345c2b7b8accc53b7657d/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:40.922108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e931a8708f48eb962baa3f93a7cd17, user_id=8079ddf9ea8345c2b7b8accc53b7657d, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:40.924447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:40.967308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 854/1690] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:40 2022] PUT /identity/v3/projects/92e931a8708f48eb962baa3f93a7cd17/users/8079ddf9ea8345c2b7b8accc53b7657d/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:40.974267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:41.001506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dcefa-9079-401e-8369-e4519847e09b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:41.100375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:41.278353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:41.279198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:41.279491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:41.279767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:41.280492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:41.287240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88094eb9-a33e-4d95-930d-94d43bf707c2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:41.353475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:41.359263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 837/1691] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:41.370458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e479-25b6-46a0-b79e-482a447b9d23 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:41.405032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:41.405830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:41.406127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:41.406409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] PATH_INFO: `/v3/projects/92e931a8708f48eb962baa3f93a7cd17/users/8079ddf9ea8345c2b7b8accc53b7657d/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:41.428102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e931a8708f48eb962baa3f93a7cd17, user_id=8079ddf9ea8345c2b7b8accc53b7657d, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:41.444914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:41.490589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 855/1692] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:41 2022] PUT /identity/v3/projects/92e931a8708f48eb962baa3f93a7cd17/users/8079ddf9ea8345c2b7b8accc53b7657d/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:41.503099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e33d6-b630-45f5-b1f5-c3b9de6ec605 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:41.504169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e33d6-b630-45f5-b1f5-c3b9de6ec605 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:41.504611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e33d6-b630-45f5-b1f5-c3b9de6ec605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:41.574591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1a0e33d6-b630-45f5-b1f5-c3b9de6ec605 None None] MFA Rules not processed for user `8079ddf9ea8345c2b7b8accc53b7657d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:41.576203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1a0e33d6-b630-45f5-b1f5-c3b9de6ec605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:41.871673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 838/1693] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:41.880074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:41.882361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:42.092307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441624-8868-4400-bcc5-d04094644ff2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:42.092861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441624-8868-4400-bcc5-d04094644ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:42.094439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441624-8868-4400-bcc5-d04094644ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:42.130146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a32e2f-f075-4958-88d4-35cf03f89fdd admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:42.153991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:42.155180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:42.155705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:42.156173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:42.167525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c5441624-8868-4400-bcc5-d04094644ff2 None None] MFA Rules not processed for user `a6f5e471a7ac4218a35ffe1840d91575`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:42.170345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:42.171170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c5441624-8868-4400-bcc5-d04094644ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:42.174727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:42.212105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 856/1694] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:42.361172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 839/1695] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:42.388312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:42.398007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:42.413750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad6e-ca01-47e6-86f2-a7a63e55466c service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:42.425770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:42.426456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:42.426734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:42.427175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:42.611687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:42.621801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697326-b6ac-47cd-844d-e4a32b571f5c service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:42.647817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 857/1696] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:45.246798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e61f-456a-414d-be39-26e24d520146 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:45.247380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e61f-456a-414d-be39-26e24d520146 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:45.247884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e61f-456a-414d-be39-26e24d520146 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:45.250159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 840/1697] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:11:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:45.517722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2167cb-3f0e-4db1-8ed6-1f9ff0784ca5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:45.519857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2167cb-3f0e-4db1-8ed6-1f9ff0784ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:45.520503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2167cb-3f0e-4db1-8ed6-1f9ff0784ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:45.607439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5a2167cb-3f0e-4db1-8ed6-1f9ff0784ca5 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:45.632724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5a2167cb-3f0e-4db1-8ed6-1f9ff0784ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:45.826827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:45.889778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 858/1698] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:46.949670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:46.950646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:46.951184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:46.952850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 841/1699] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:11:46.969121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:46.970149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:46.970681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:47.078754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:47.080046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:47.123024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 859/1700] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:11:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:49.423651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:49.426836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:49.489403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a66233-e5bf-42d7-88ed-d8f9f983d810 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:49.503420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:49.504070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:49.504992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:49.505277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:49.537620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:49.539342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:49.554467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 842/1701] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:49.775240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:49.776820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:49.800724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74831d-ae52-4860-bb15-091d04aa53b7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:49.819024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:49.819890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:49.820565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:49.820982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:49.842733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:49.844970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:49.875123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 860/1702] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:49.908807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:49.910780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:49.992782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:50.103891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d59b3-8bb6-4bdb-b7cd-eef3b92d6586 service nova] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:50.115822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:50.116862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.121048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.121048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] PATH_INFO: `/v3/users/236fae7fbbc242478c679e9459d2111b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.122434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236fae7fbbc242478c679e9459d2111b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:50.128945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:50.225987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236fae7fbbc242478c679e9459d2111b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.226807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236fae7fbbc242478c679e9459d2111b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.244944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236fae7fbbc242478c679e9459d2111b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.270396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236fae7fbbc242478c679e9459d2111b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.310107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:50.319516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4a714b67-fdfd-4afa-9296-705e3ee6f487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236fae7fbbc242478c679e9459d2111b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.339426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a921d63-11e8-4904-b560-dc8df901e655 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:50.362222 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:50.365466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.366325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 843/1703] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:49 2022] DELETE /identity/v3/users/236fae7fbbc242478c679e9459d2111b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:50.370111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.372337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] PATH_INFO: `/v3/users/d0ad3ddc76664d94a82e270e44c74a8f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.376420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.376712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.376994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.377956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ad3ddc76664d94a82e270e44c74a8f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:50.383778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:50.409228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:50.410658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:50.501574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ad3ddc76664d94a82e270e44c74a8f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.533676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ad3ddc76664d94a82e270e44c74a8f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.556953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ad3ddc76664d94a82e270e44c74a8f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.557795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ad3ddc76664d94a82e270e44c74a8f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.558455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc32c1c4-1fcb-4b84-8795-1be1be629d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ad3ddc76664d94a82e270e44c74a8f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:50.579765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 844/1704] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:50.592904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:50.594513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:50.594898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 861/1705] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:50 2022] DELETE /identity/v3/users/d0ad3ddc76664d94a82e270e44c74a8f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:50.600145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.600729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.601270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.639494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:50.641904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:50.718901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:50.736982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe3ed-bdc6-4d9d-8125-2f941de21f55 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:50.764589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 862/1706] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:50.778045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:50.853442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:50.862851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:50.863538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.863820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.864096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.891516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:50.892984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:50.903912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3c1ec-0216-4cc3-b031-a84324c38ec7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:50.908676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:50.909616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:50.910252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:50.910727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:50.934069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 845/1707] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:50.939662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:50.942480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:50.959820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 863/1708] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:11:51.588030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:51.589986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:51.600136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:51.601576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:51.607425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59a7d7-2976-4b54-9108-a060fea1b15f service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:51.614519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:51.615181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:51.616716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:51.616961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] PATH_INFO: `/v3/projects/7aaf9e3ae3b2410b915efb52c52a4ab9` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:51.619159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aaf9e3ae3b2410b915efb52c52a4ab9)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:51.621098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:51.622926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7bfe4-c7d3-4300-8647-9f0c39be5780 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:51.633103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:51.633925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:51.634443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:51.634736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] PATH_INFO: `/v3/projects/febb9e5ae0044129914cb4281642a68c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:51.635946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=febb9e5ae0044129914cb4281642a68c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:51.636972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: WARNING py.warnings [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 09:11:51.636972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: warnings.warn(msg) Jul 28 09:11:51.636972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: Jul 28 09:11:51.638347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:51.693936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aaf9e3ae3b2410b915efb52c52a4ab9', 'request_id': 'req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.703448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aaf9e3ae3b2410b915efb52c52a4ab9', 'request_id': 'req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.713110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aaf9e3ae3b2410b915efb52c52a4ab9', 'request_id': 'req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.716506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febb9e5ae0044129914cb4281642a68c', 'request_id': 'req-1e06b494-1ef0-4399-a005-80d5c382f6ca'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.725677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febb9e5ae0044129914cb4281642a68c', 'request_id': 'req-1e06b494-1ef0-4399-a005-80d5c382f6ca'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.726838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] The token cache is being invalidate because project 7aaf9e3ae3b2410b915efb52c52a4ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:51.727137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.727916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.735362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 846/1709] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:51 2022] DELETE /identity/v3/projects/7aaf9e3ae3b2410b915efb52c52a4ab9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:51.739888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:51.741577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:51.744003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febb9e5ae0044129914cb4281642a68c', 'request_id': 'req-1e06b494-1ef0-4399-a005-80d5c382f6ca'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.757407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] The token cache is being invalidate because project febb9e5ae0044129914cb4281642a68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:51.758134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.758948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.760878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 864/1710] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:51 2022] DELETE /identity/v3/projects/febb9e5ae0044129914cb4281642a68c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:51.765047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:51.771053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:51.823769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30abb4-a8a0-4815-a31d-39dc2f88ea93 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:51.842454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06b494-1ef0-4399-a005-80d5c382f6ca admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:51.848695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:51.849521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:51.849979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:51.850397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] PATH_INFO: `/v3/users/8079ddf9ea8345c2b7b8accc53b7657d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:51.851728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8079ddf9ea8345c2b7b8accc53b7657d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:51.854033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:51.867694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:51.899293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:51.900029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:51.900315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:51.900880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] PATH_INFO: `/v3/users/4694ab3e89b742e4b7741d45407f4297` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:51.902007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4694ab3e89b742e4b7741d45407f4297)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:51.904099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:51.940783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8079ddf9ea8345c2b7b8accc53b7657d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.953492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:51.956154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8079ddf9ea8345c2b7b8accc53b7657d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.968153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8079ddf9ea8345c2b7b8accc53b7657d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.969349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8079ddf9ea8345c2b7b8accc53b7657d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.970041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8079ddf9ea8345c2b7b8accc53b7657d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.984853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4694ab3e89b742e4b7741d45407f4297'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.985716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 865/1711] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:51 2022] DELETE /identity/v3/users/8079ddf9ea8345c2b7b8accc53b7657d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:51.985886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4694ab3e89b742e4b7741d45407f4297'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:51.998681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4694ab3e89b742e4b7741d45407f4297'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.007201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4694ab3e89b742e4b7741d45407f4297'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.016515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4694ab3e89b742e4b7741d45407f4297'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.024060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 847/1712] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:51 2022] DELETE /identity/v3/users/4694ab3e89b742e4b7741d45407f4297 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:52.430178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.431827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.441693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.443474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.519733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30675b2-9d5c-4927-b244-7509b88c150f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.525033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.525795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.526229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.526588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] PATH_INFO: `/v3/projects/92e931a8708f48eb962baa3f93a7cd17` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.528205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e931a8708f48eb962baa3f93a7cd17)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.530283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.574366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f48e8-8e8c-4fb3-a7b5-ddf1f7764a22 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.580951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.582108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.582717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.582997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] PATH_INFO: `/v3/projects/8bba812cf9734deaba52757594a4e5aa` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.585182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bba812cf9734deaba52757594a4e5aa)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.588591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.618012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e931a8708f48eb962baa3f93a7cd17', 'request_id': 'req-8f04e050-0c06-4b9f-bec3-ca753b7939dc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.631499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e931a8708f48eb962baa3f93a7cd17', 'request_id': 'req-8f04e050-0c06-4b9f-bec3-ca753b7939dc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.634304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:52.647309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e931a8708f48eb962baa3f93a7cd17', 'request_id': 'req-8f04e050-0c06-4b9f-bec3-ca753b7939dc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.656460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bba812cf9734deaba52757594a4e5aa', 'request_id': 'req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.666996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] The token cache is being invalidate because project 92e931a8708f48eb962baa3f93a7cd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:52.667571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.674395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8f04e050-0c06-4b9f-bec3-ca753b7939dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.674670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bba812cf9734deaba52757594a4e5aa', 'request_id': 'req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.676116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 848/1713] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:52 2022] DELETE /identity/v3/projects/92e931a8708f48eb962baa3f93a7cd17 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:52.678197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306e8-052d-41e9-bcd8-49ba7c10c1b9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.679028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306e8-052d-41e9-bcd8-49ba7c10c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.679846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306e8-052d-41e9-bcd8-49ba7c10c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.681072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bba812cf9734deaba52757594a4e5aa', 'request_id': 'req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.696467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] The token cache is being invalidate because project 8bba812cf9734deaba52757594a4e5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:52.697122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.698150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf2b1463-a387-4ca2-89b9-51734f5dc3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:52.700398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 866/1714] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:52 2022] DELETE /identity/v3/projects/8bba812cf9734deaba52757594a4e5aa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:52.704859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159f6d7-ee1b-4cc1-8911-a68abe1f9f7f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.705139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159f6d7-ee1b-4cc1-8911-a68abe1f9f7f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.705411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159f6d7-ee1b-4cc1-8911-a68abe1f9f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.736001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d159f6d7-ee1b-4cc1-8911-a68abe1f9f7f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:52.738674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d159f6d7-ee1b-4cc1-8911-a68abe1f9f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.742768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-626306e8-052d-41e9-bcd8-49ba7c10c1b9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:52.744528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-626306e8-052d-41e9-bcd8-49ba7c10c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.822289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 867/1715] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:52.830359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.830682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 849/1716] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:52.831536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.831819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.838473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.838976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.839419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.856264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:52.857756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.861250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:52.863165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:52.868719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 868/1717] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:52.874551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 850/1718] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:52.874693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.875598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.880824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:52.885441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846f82b-4857-4298-a4b8-bbcc477b8f6c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.887632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82b52-a681-41d4-8822-9f061c4377f7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.890391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.891437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.891831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.892772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.894071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:52.894399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.896228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.896362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.897159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.897512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.897894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.898657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:52.898975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.900587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.914208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 851/1719] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:52.915713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.920068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 869/1720] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:11:52.928629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a37129-6deb-44cc-af97-5f0707b7fbbd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.929408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:52.933381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.935397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.935397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.935397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.935397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.937326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.940596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898817de-5d31-4815-9efb-99db979f70c7 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:52.958211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:52.959201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:52.959584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:52.959911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:52.960632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:52.962686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:52.976211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 852/1721] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:52.977799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.000068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 870/1722] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:53.001161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.079354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062da65e-f44f-4ad3-8744-097034b0aa14 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.081172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454586-0a0b-4074-9f3b-f33ea79eb981 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.085556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.086391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.086735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.087079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.087907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.088483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.090184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.090549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.091001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.091442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.092441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.094767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.124258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 871/1723] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:53.125872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.140460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 853/1724] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:52 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:53.141884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.142810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c02e4d-d751-4b36-a1a2-c1b73753b3be admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.150153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.151196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.151731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.152203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.153102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.155397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.172020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afe55-e36f-4f8d-be43-52c3434bae86 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.172917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 872/1725] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.174098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.182920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.183650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.184407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.184693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.185475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.187204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.187316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ab383-142e-4e1e-8c10-437c07639079 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.200244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.201076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.201531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.201995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.202857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.204790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.215178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 854/1726] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.219357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.225478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 873/1727] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.226294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.231193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274263cc-3236-4d15-a1ad-32cf64d7eea5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.237316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.238613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.239162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.239535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.241388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.243159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.245648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debd7c-125e-49ae-9efc-99dd1d3296f8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.251128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.251989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.252434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.252847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] PATH_INFO: `/v3/projects/82f99318d5884cf2bcc3bbe971b46dce/users/31b5258d390f4c0288b77d87b4f45bbf/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.260605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 855/1728] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.261801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.270884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f99318d5884cf2bcc3bbe971b46dce, user_id=31b5258d390f4c0288b77d87b4f45bbf, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.273928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00321f62-2913-49b2-b8ee-6985edbd7bed admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.277519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86244315-1444-4679-8f92-11d950dd5b64 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.283439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.284342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.286369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.286369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] PATH_INFO: `/v3/projects/b6c6bc50a4ed46679dd05c143fe6daf6/users/bc071df5074f4c769be0968c17fe1a0d/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.302326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 874/1729] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:53 2022] PUT /identity/v3/projects/82f99318d5884cf2bcc3bbe971b46dce/users/31b5258d390f4c0288b77d87b4f45bbf/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:53.304119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.304413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.304688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.316832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:53.333091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c6bc50a4ed46679dd05c143fe6daf6, user_id=bc071df5074f4c769be0968c17fe1a0d, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.341392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] MFA Rules not processed for user `31b5258d390f4c0288b77d87b4f45bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:53.341569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d65515-3ca9-4035-8bb5-bd9678d1d550 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.343015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:53.383784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 856/1730] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:53 2022] PUT /identity/v3/projects/b6c6bc50a4ed46679dd05c143fe6daf6/users/bc071df5074f4c769be0968c17fe1a0d/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:53.387145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030db1-33f8-4239-8915-18499b2e177d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.387508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030db1-33f8-4239-8915-18499b2e177d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.387876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030db1-33f8-4239-8915-18499b2e177d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.415717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:53.419819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-be030db1-33f8-4239-8915-18499b2e177d None None] MFA Rules not processed for user `bc071df5074f4c769be0968c17fe1a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:53.421159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-be030db1-33f8-4239-8915-18499b2e177d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:53.423752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 875/1731] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:53.427580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.504536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba3aee-0de3-41d4-92e2-442bb19262b7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.511016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 857/1732] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:11:53.511929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030db1-33f8-4239-8915-18499b2e177d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.574587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.575207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.575488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.575758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.576423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.578494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.593208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030db1-33f8-4239-8915-18499b2e177d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.598873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.599647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.600094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.600439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.601215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.603392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.619686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 876/1733] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:53.620820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.664515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 858/1734] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:53.668531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.749905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d88d8-d04a-4a29-a1cb-a05b848e0def admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.766583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.767151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.767436 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.767729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.768925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.770425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.771255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d34a0-8519-4121-bd7b-5dafbf63844b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.778502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.779423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.780129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.780617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.781521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.783705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.795613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:53.823763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 859/1736] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:11:53.825107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.829938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 877/1736] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:53 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:53.832240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.835891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b67d0c-b47e-4930-b1e4-056f27f71e59 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.841989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.842619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.844636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.844946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.845670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.848101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.849828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9de88-1d0d-4520-a6fb-9228ecaa005b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.855335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.856240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.857319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.858339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.859331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.861347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.889392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 860/1737] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.890684 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.904242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 878/1738] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:53.913437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:53.933337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d40b2-4b5c-47f1-9053-10522be930d5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.940045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.941567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.942038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.942426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] PATH_INFO: `/v3/projects/848d5a10b31f47f4830f8349611149b6/users/55c42ac9cd1446e193bee94a7f903584/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.945970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980ff7d-dcab-4615-bacc-203bd14cc492 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:53.952353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:53.953334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:53.953624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:53.954573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] PATH_INFO: `/v3/projects/b68a1b064b4c4a05baa97062dce5087d/users/0060ae7c045c4601882951a0d4d30cd7/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:53.963665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=848d5a10b31f47f4830f8349611149b6, user_id=55c42ac9cd1446e193bee94a7f903584, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.967075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:53.969221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:53.981702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68a1b064b4c4a05baa97062dce5087d, user_id=0060ae7c045c4601882951a0d4d30cd7, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:53.984977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.001864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 861/1739] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:53 2022] PUT /identity/v3/projects/848d5a10b31f47f4830f8349611149b6/users/55c42ac9cd1446e193bee94a7f903584/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:54.001864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.004413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 879/1740] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:53 2022] PUT /identity/v3/projects/b68a1b064b4c4a05baa97062dce5087d/users/0060ae7c045c4601882951a0d4d30cd7/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:54.008104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.087566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4a856-ebaa-42db-924d-7d3c767f5389 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.094495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.095411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.095938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.096208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.097664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.102899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.112151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4bcab-6314-4096-96c9-1ff2af7abbac admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.124502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 880/1741] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:11:54.125972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.128498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.130173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.133032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.133032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.134119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.135936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.149634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b4a71-cdbc-4937-a8e9-f9b2abece7f8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.156221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.157001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.157768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.158154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] PATH_INFO: `/v3/projects/b68a1b064b4c4a05baa97062dce5087d/users/0060ae7c045c4601882951a0d4d30cd7/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.161812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68a1b064b4c4a05baa97062dce5087d, user_id=0060ae7c045c4601882951a0d4d30cd7, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.165022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 862/1742] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:54.165163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959b352-983a-4cc8-98e5-aa01f4c0f1c5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.166158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.178358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 881/1743] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:54 2022] PUT /identity/v3/projects/b68a1b064b4c4a05baa97062dce5087d/users/0060ae7c045c4601882951a0d4d30cd7/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:54.179084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7513-810a-4d9a-8e46-2aba2d2238dc admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.190516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.190915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.191272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.230877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] MFA Rules not processed for user `0060ae7c045c4601882951a0d4d30cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:54.232325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:54.278835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.279578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.279867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.280149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] PATH_INFO: `/v3/projects/848d5a10b31f47f4830f8349611149b6/users/55c42ac9cd1446e193bee94a7f903584/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.282432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=848d5a10b31f47f4830f8349611149b6, user_id=55c42ac9cd1446e193bee94a7f903584, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.286009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956182-2351-4960-872e-d15a440dcc4d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.309702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 863/1744] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:54 2022] PUT /identity/v3/projects/848d5a10b31f47f4830f8349611149b6/users/55c42ac9cd1446e193bee94a7f903584/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:54.311283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.311567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.311842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.341754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 882/1745] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:54.345960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] MFA Rules not processed for user `55c42ac9cd1446e193bee94a7f903584`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:54.347890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:54.355139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.356428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:54.389545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:54.422127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4197bcc-cac8-40e0-b133-f32782a89808 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.437357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:54.451851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 864/1746] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:54.466088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.467521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:54.499622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.500262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.500546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.500826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.539352 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023573dc-6ad3-4644-82df-201b61c5ab60 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.546087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.547327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.547729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.547904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.561475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.563047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.573520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 865/1747] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:11:54.580739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.582297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.593374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 883/1748] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:11:54.645862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.782608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:54.790533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f5c7b-9fe6-46f9-b545-ac5e440ed06f service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.795510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779c5b-4eca-4cd1-86c4-3221d6feccab service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:54.797632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.798572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.798681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.798944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] PATH_INFO: `/v3/users/31b5258d390f4c0288b77d87b4f45bbf` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.800768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b5258d390f4c0288b77d87b4f45bbf)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.803474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.804289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:54.805319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.806571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.807069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.876823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b5258d390f4c0288b77d87b4f45bbf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:54.877601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b5258d390f4c0288b77d87b4f45bbf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:54.885310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:54.887786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52967853-a199-4689-97a4-afd1fd1689af service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:54.892641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b5258d390f4c0288b77d87b4f45bbf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:54.903060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 884/1749] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:54.905778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b5258d390f4c0288b77d87b4f45bbf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:54.912602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:54.917148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b5258d390f4c0288b77d87b4f45bbf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:54.930887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 866/1750] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:54 2022] DELETE /identity/v3/users/31b5258d390f4c0288b77d87b4f45bbf => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:54.935229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:54.935586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:54.935827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:54.976569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:54.980219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:55.061216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:55.073491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 885/1751] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:55.107726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:55.109156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:55.184974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4090-b862-47cf-912a-ebe39955d8fb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:55.192080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:55.193764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:55.194384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:55.194693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:55.209264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:55.210811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:55.224764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 867/1752] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:55.694976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:55.696422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:55.708654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44ca3c-8bd6-4db2-895b-243a6e660b41 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:55.713173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:55.713824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:55.714155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:55.714433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] PATH_INFO: `/v3/projects/82f99318d5884cf2bcc3bbe971b46dce` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:55.715585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f99318d5884cf2bcc3bbe971b46dce)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:55.717729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:55.771747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f99318d5884cf2bcc3bbe971b46dce', 'request_id': 'req-1392fa83-ad11-461c-bd6b-66e775d1dacc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.780180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f99318d5884cf2bcc3bbe971b46dce', 'request_id': 'req-1392fa83-ad11-461c-bd6b-66e775d1dacc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.786912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f99318d5884cf2bcc3bbe971b46dce', 'request_id': 'req-1392fa83-ad11-461c-bd6b-66e775d1dacc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.797114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] The token cache is being invalidate because project 82f99318d5884cf2bcc3bbe971b46dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:55.797397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.798060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.799571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 886/1753] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:55 2022] DELETE /identity/v3/projects/82f99318d5884cf2bcc3bbe971b46dce => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:55.804071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:55.813791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fbc31-5434-476b-9eba-3e08443980d2 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:55.868868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:55.869421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:55.869698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:55.870016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] PATH_INFO: `/v3/users/0060ae7c045c4601882951a0d4d30cd7` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:55.871157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0060ae7c045c4601882951a0d4d30cd7)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:55.873263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:55.918033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0060ae7c045c4601882951a0d4d30cd7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.918823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0060ae7c045c4601882951a0d4d30cd7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.931963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0060ae7c045c4601882951a0d4d30cd7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.941359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0060ae7c045c4601882951a0d4d30cd7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.948487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fd1bf7a9-96cc-4762-9da6-546c7ebef7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0060ae7c045c4601882951a0d4d30cd7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:55.954386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 868/1754] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:11:55 2022] DELETE /identity/v3/users/0060ae7c045c4601882951a0d4d30cd7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:56.230573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.232113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:56.245004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fa83-ad11-461c-bd6b-66e775d1dacc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.302471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.303120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.303399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.303669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] PATH_INFO: `/v3/projects/b68a1b064b4c4a05baa97062dce5087d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.304825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68a1b064b4c4a05baa97062dce5087d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.306792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.356012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68a1b064b4c4a05baa97062dce5087d', 'request_id': 'req-2f13ec6b-2a36-498f-9298-87b99cd84678'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:56.365102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68a1b064b4c4a05baa97062dce5087d', 'request_id': 'req-2f13ec6b-2a36-498f-9298-87b99cd84678'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:56.374795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68a1b064b4c4a05baa97062dce5087d', 'request_id': 'req-2f13ec6b-2a36-498f-9298-87b99cd84678'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:56.386481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] The token cache is being invalidate because project b68a1b064b4c4a05baa97062dce5087d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:11:56.386769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:56.387453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f13ec6b-2a36-498f-9298-87b99cd84678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:11:56.389425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 887/1755] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:11:56 2022] DELETE /identity/v3/projects/b68a1b064b4c4a05baa97062dce5087d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:56.394815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.395084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.395358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.418067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:56.419142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:56.429176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:56.488564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 869/1756] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:56.496659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.497563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.497563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.519368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:56.521961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:56.529294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 888/1757] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:56.533509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.544362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d7b1d-22a3-49a6-beeb-25feb0895793 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.548984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.549617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.549919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.550258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.555182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:11:56.555182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.555182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.566785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 870/1758] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:11:56.571552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.582340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab5eba-56c3-488f-80d3-6d556960ab28 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.587346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.588004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.588284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.588559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.589248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.591260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.596221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:56.632917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.633989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 889/1759] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:56.645415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5ca70-bc84-40d2-a142-e38d2156e015 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.718212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.718836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.719115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.719386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.720043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.722015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.747510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 871/1760] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:56.748605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.759530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230a477-23f0-4fb7-8743-87d00722eede admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.764151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.764765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.765044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.765313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.766141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.767849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.781874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 890/1761] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:56.786553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.801730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e58e5-fc38-414e-b49f-906ff023770c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.806826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.807513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.807842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.808123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.808861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.811145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729550b-9ff5-4b6a-a8d0-799b5fbfa7de admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.837879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 872/1762] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:11:56.839368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:56.850955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f044a-21fb-41a4-942e-12256d50dc8c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:56.855838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:56.856592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.856947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.857326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] PATH_INFO: `/v3/projects/8a583c7e90c94e09b1ecc03dd60bbb05/users/4d4ac405abbf4af0b1cf82f432793115/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.872671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a583c7e90c94e09b1ecc03dd60bbb05, user_id=4d4ac405abbf4af0b1cf82f432793115, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:56.875373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:56.889361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 891/1763] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:56 2022] PUT /identity/v3/projects/8a583c7e90c94e09b1ecc03dd60bbb05/users/4d4ac405abbf4af0b1cf82f432793115/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:56.900177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:56.900465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:56.900738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:56.928037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] MFA Rules not processed for user `4d4ac405abbf4af0b1cf82f432793115`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:56.929259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:56.992694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 873/1764] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:56.997134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.057559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fb97-d923-4fd8-bb31-89281b9929b0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.062202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.062822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.063102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.063379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.064048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.066544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.092321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 892/1765] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:11:56 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:57.096375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.106869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453c38b-98fa-4298-8833-67b7c0de37ca None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.114856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:57.163244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.163873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.164154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.164429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.165097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.167066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.188461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 874/1766] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:11:57 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:11:57.193098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.203678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e2730-c170-4219-ac19-1b129e2c9a27 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.208284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.208903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.209189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.209469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.210215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.211820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.222412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 893/1767] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:57.226405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.236070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92a9d9-323c-4514-8823-5e951e718f9d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.241125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.241777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.242085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.242362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] PATH_INFO: `/v3/projects/77652b6c6ae640dca95f8908c408ed6c/users/6da4bad51e454e378b645807bfbab8bc/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.257205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77652b6c6ae640dca95f8908c408ed6c, user_id=6da4bad51e454e378b645807bfbab8bc, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.259818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.272110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 875/1768] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:57 2022] PUT /identity/v3/projects/77652b6c6ae640dca95f8908c408ed6c/users/6da4bad51e454e378b645807bfbab8bc/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:11:57.276124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.286248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17310e6a-93ae-4e32-820d-a09bda53afe0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.342460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.343088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.343360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.343631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.344322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.345936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a9a66-a45b-4592-b726-1c94c3d80e96 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.357031 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 894/1769] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:11:57.361089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.375465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f978f-37f2-4d95-b670-f67cc3016638 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.382966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.383645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.383928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.384206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] PATH_INFO: `/v3/projects/77652b6c6ae640dca95f8908c408ed6c/users/6da4bad51e454e378b645807bfbab8bc/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.386202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77652b6c6ae640dca95f8908c408ed6c, user_id=6da4bad51e454e378b645807bfbab8bc, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.388818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.399129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 876/1770] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:11:57 2022] PUT /identity/v3/projects/77652b6c6ae640dca95f8908c408ed6c/users/6da4bad51e454e378b645807bfbab8bc/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:11:57.411090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.411381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.411660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.441556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] MFA Rules not processed for user `6da4bad51e454e378b645807bfbab8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:11:57.442829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:57.452806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:11:57.530030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 895/1771] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:11:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:11:57.543558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.545821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:57.619445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c2e6-438c-4116-a76e-6c52322d9d70 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.624301 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.625054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.625776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.625776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.637063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.638576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.649035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 877/1772] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:11:57.712545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:11:57.713973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:11:57.725664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5dd7-9d75-4750-9e2d-6f9c6fe9df29 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:11:57.729323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:11:57.730011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:11:57.730323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:11:57.730775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:11:57.789060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:11:57.790470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaa5a2-dbae-4469-8846-2cf9c0fa3911 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:11:57.799352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 896/1773] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:11:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:01.871079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:01.872803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:01.968633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62f371-966d-413e-98b7-dd7b8b2b844c service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:01.973982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:01.974718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:01.975241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:01.975522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] PATH_INFO: `/v3/users/bc071df5074f4c769be0968c17fe1a0d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:01.976849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc071df5074f4c769be0968c17fe1a0d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:01.978977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:01.988941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:02.045290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc071df5074f4c769be0968c17fe1a0d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.046732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc071df5074f4c769be0968c17fe1a0d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.066518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc071df5074f4c769be0968c17fe1a0d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.078297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc071df5074f4c769be0968c17fe1a0d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.088415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc071df5074f4c769be0968c17fe1a0d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.095440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 878/1774] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:01 2022] DELETE /identity/v3/users/bc071df5074f4c769be0968c17fe1a0d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:02.099867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.100430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.101114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.133742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:02.137126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:02.233265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 897/1775] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:02.241767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.243629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:02.334869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7bd50-83d7-48bc-90fc-d96297bb4e25 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.343796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.344507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.344847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.345127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.360283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.361834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.373728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 879/1776] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:02.649605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.651108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:02.666743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69f134-b584-4a6f-a288-98e27ba0d72c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.671946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.672717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.673059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.673457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] PATH_INFO: `/v3/projects/b6c6bc50a4ed46679dd05c143fe6daf6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.674798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c6bc50a4ed46679dd05c143fe6daf6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.677566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.717067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:02.726287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c6bc50a4ed46679dd05c143fe6daf6', 'request_id': 'req-044edaf7-35d4-4f8e-be93-dfbe22e2d720'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.734116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c6bc50a4ed46679dd05c143fe6daf6', 'request_id': 'req-044edaf7-35d4-4f8e-be93-dfbe22e2d720'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.741360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c6bc50a4ed46679dd05c143fe6daf6', 'request_id': 'req-044edaf7-35d4-4f8e-be93-dfbe22e2d720'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.752109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] The token cache is being invalidate because project b6c6bc50a4ed46679dd05c143fe6daf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:02.752396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.753085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.754697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 898/1777] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:02 2022] DELETE /identity/v3/projects/b6c6bc50a4ed46679dd05c143fe6daf6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:02.758740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:02.768741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c49f7-4912-4a1a-a431-0030acb9aff7 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:02.824271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:02.824918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:02.825205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:02.825487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] PATH_INFO: `/v3/users/55c42ac9cd1446e193bee94a7f903584` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:02.826682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c42ac9cd1446e193bee94a7f903584)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:02.828617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:02.866333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:02.879521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c42ac9cd1446e193bee94a7f903584'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.880181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c42ac9cd1446e193bee94a7f903584'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.892232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c42ac9cd1446e193bee94a7f903584'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.899672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c42ac9cd1446e193bee94a7f903584'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.909273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a4118090-1584-47c7-b69b-b4996553b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c42ac9cd1446e193bee94a7f903584'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:02.915992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 880/1778] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:02 2022] DELETE /identity/v3/users/55c42ac9cd1446e193bee94a7f903584 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:03.236213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.238399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:03.262124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edaf7-35d4-4f8e-be93-dfbe22e2d720 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.340372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.341228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.341570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.341944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] PATH_INFO: `/v3/projects/848d5a10b31f47f4830f8349611149b6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.343324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848d5a10b31f47f4830f8349611149b6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.345244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.395458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848d5a10b31f47f4830f8349611149b6', 'request_id': 'req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.404147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848d5a10b31f47f4830f8349611149b6', 'request_id': 'req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.412334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848d5a10b31f47f4830f8349611149b6', 'request_id': 'req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.427149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] The token cache is being invalidate because project 848d5a10b31f47f4830f8349611149b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:03.427567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.428457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ddbbb4a3-dc6c-4881-aedc-9a0875aaf761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.430333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 899/1779] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:03 2022] DELETE /identity/v3/projects/848d5a10b31f47f4830f8349611149b6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:03.435453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.435827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.436007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.459461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:03.460691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:03.543756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 881/1780] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:03.551840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.551840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.551840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.572805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:03.575126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:03.584327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 900/1781] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:03.588535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.600690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1ac7f-0200-4ef4-8bba-fd7e58fed52f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.605606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.606289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.606570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.606841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.607536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:03.607902 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.609616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.624635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 882/1782] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:03.629566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.642177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b1130-8bf5-46dd-b6a3-bb67d04388f7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.648412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.649298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.649880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.650239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.650993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.653114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.695238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 901/1783] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:03 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:03.696572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.715654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.719469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:03.721168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc191a9f-0d15-40da-9c9a-3f56b7277b7d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.770862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:03.804186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.805461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.806245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.807193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.808171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.810390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.814036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfbe48-51c6-4aea-a87d-08344d0ca7b3 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.818777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.819776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.819776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.819990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] PATH_INFO: `/v3/users/4d4ac405abbf4af0b1cf82f432793115` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.822158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d4ac405abbf4af0b1cf82f432793115)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.826111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.841088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 883/1784] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:03 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:03.842196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.853743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e81da-6821-4e35-ad29-1d15f251c7cf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.860004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.861062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.861291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.861617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.862338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.863991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.879275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 884/1785] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:03.880302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.890798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b56597-639e-4713-87d6-e1d0e0eef80a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.895036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d4ac405abbf4af0b1cf82f432793115'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.897019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.897667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.898005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.898296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.899005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.900643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.912144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 885/1786] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:03.912633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d4ac405abbf4af0b1cf82f432793115'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.916699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:03.927435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:03.927699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d4ac405abbf4af0b1cf82f432793115'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.928354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d4ac405abbf4af0b1cf82f432793115'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.929234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c2b9cb9c-0fdc-4a7b-aee4-758995b50348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d4ac405abbf4af0b1cf82f432793115'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:03.937395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725583fd-8522-4c69-aab9-97dbfdc46bc1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:03.944643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:03.945452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.946074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.946418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] PATH_INFO: `/v3/projects/47a116cb8061487c88930ad3db813b6d/users/a329a3c5c135413fa2aea728deeba719/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.946739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 902/1787] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:03 2022] DELETE /identity/v3/users/4d4ac405abbf4af0b1cf82f432793115 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:03.951406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38790-892c-4560-9477-03aa3295df63 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:03.951691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38790-892c-4560-9477-03aa3295df63 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:03.952063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38790-892c-4560-9477-03aa3295df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:03.971977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a116cb8061487c88930ad3db813b6d, user_id=a329a3c5c135413fa2aea728deeba719, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:03.972235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-40d38790-892c-4560-9477-03aa3295df63 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:03.973943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-40d38790-892c-4560-9477-03aa3295df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:03.976107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb718d84-dc78-4625-94a1-2d02cac0cd67 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:03.994837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 886/1788] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:03 2022] PUT /identity/v3/projects/47a116cb8061487c88930ad3db813b6d/users/a329a3c5c135413fa2aea728deeba719/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:04.004988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.005279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.005558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.034623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] MFA Rules not processed for user `a329a3c5c135413fa2aea728deeba719`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:04.036412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:04.057324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 903/1789] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:04.067576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38790-892c-4560-9477-03aa3295df63 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.068959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-40d38790-892c-4560-9477-03aa3295df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:04.111688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 887/1790] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:04.116067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.117374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:04.127352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:04.131895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38790-892c-4560-9477-03aa3295df63 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.190295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.191548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.191910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.192124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.208950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.211374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.220033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026a24-4b41-457e-a74f-0e6e1010927c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.227972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 904/1791] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:04.233289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.233976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.234262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.234562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.235377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.242285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.299734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 888/1792] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:04 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:04.303711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.319661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb982-91f1-4e40-b83c-6cd5c91b7cd8 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.426835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.427646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.428357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.428847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.429646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.432730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.465443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 905/1793] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:04 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:04.469598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.483812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11721e1f-b615-4438-9743-1665803fc5cc admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.497539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.498149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.498439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.498874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.499595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.501236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.531600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 889/1794] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:04.536059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.549737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e8e7-2fbe-41a0-a327-524de4f8b9ee admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.564168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.565376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.566531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.567057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] PATH_INFO: `/v3/projects/fdf8a07539c94726bd5e5d4b846e479e/users/b6c1f8b8286a4c80a71daac1ead0bea8/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.608265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf8a07539c94726bd5e5d4b846e479e, user_id=b6c1f8b8286a4c80a71daac1ead0bea8, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.611710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.669972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 906/1795] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:04 2022] PUT /identity/v3/projects/fdf8a07539c94726bd5e5d4b846e479e/users/b6c1f8b8286a4c80a71daac1ead0bea8/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:04.674716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.691948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7e8a5-8319-48e3-8865-123466d0b704 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.812307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:04.855482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.856395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.856763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.857115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.857867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.859617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1493d-4fc2-4502-b1a9-74a3bec3aa3a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.873182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 890/1796] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:04.877170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:04.900027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceda4e3-c8e8-45f7-97fa-a444712279c4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:04.918040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:04.918763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.919296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.919443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] PATH_INFO: `/v3/projects/fdf8a07539c94726bd5e5d4b846e479e/users/b6c1f8b8286a4c80a71daac1ead0bea8/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.927918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf8a07539c94726bd5e5d4b846e479e, user_id=b6c1f8b8286a4c80a71daac1ead0bea8, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:04.932848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:04.948730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 907/1797] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:04 2022] PUT /identity/v3/projects/fdf8a07539c94726bd5e5d4b846e479e/users/b6c1f8b8286a4c80a71daac1ead0bea8/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:04.959846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:04.960393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:04.960393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:04.989116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] MFA Rules not processed for user `b6c1f8b8286a4c80a71daac1ead0bea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:04.990366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.070478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 891/1798] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:05.078492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:05.082201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.176632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c5d81-897f-40cf-ab4a-07a8473e14cb admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:05.182383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:05.183237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.183686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.184104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.203882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:05.205544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.205664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:05.208071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:05.234501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 908/1799] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:05.314851 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ca8d-898e-48be-b86c-393798d1517a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:05.319986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:05.320840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.321038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.321317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] PATH_INFO: `/v3/projects/8a583c7e90c94e09b1ecc03dd60bbb05` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.322815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a583c7e90c94e09b1ecc03dd60bbb05)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:05.325006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:05.411931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a583c7e90c94e09b1ecc03dd60bbb05', 'request_id': 'req-a89fe986-8cc2-4d44-ae29-e93d473330a9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.411931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a583c7e90c94e09b1ecc03dd60bbb05', 'request_id': 'req-a89fe986-8cc2-4d44-ae29-e93d473330a9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.418938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a583c7e90c94e09b1ecc03dd60bbb05', 'request_id': 'req-a89fe986-8cc2-4d44-ae29-e93d473330a9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.426249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:05.437716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:05.455867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:05.460603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094523f6-00e1-401d-a2cb-f5376994ba00 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:05.461318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] The token cache is being invalidate because project 8a583c7e90c94e09b1ecc03dd60bbb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:05.462071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.463384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.466602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 892/1800] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:05 2022] DELETE /identity/v3/projects/8a583c7e90c94e09b1ecc03dd60bbb05 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:05.470435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:05.473486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.540863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:05.541627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.542011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.542282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] PATH_INFO: `/v3/users/a329a3c5c135413fa2aea728deeba719` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.543590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a329a3c5c135413fa2aea728deeba719)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:05.545650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:05.559312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe986-8cc2-4d44-ae29-e93d473330a9 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:05.574297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:05.574297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.574297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.574297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] PATH_INFO: `/v3/users/6da4bad51e454e378b645807bfbab8bc` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.574629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da4bad51e454e378b645807bfbab8bc)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:05.574629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:05.613754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a329a3c5c135413fa2aea728deeba719'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.631107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a329a3c5c135413fa2aea728deeba719'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.641566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a329a3c5c135413fa2aea728deeba719'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.642443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a329a3c5c135413fa2aea728deeba719'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.643901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9a5908bc-01e7-443f-a968-23e6f0749520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a329a3c5c135413fa2aea728deeba719'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.646668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da4bad51e454e378b645807bfbab8bc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.647346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da4bad51e454e378b645807bfbab8bc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.662806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da4bad51e454e378b645807bfbab8bc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.663016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 909/1801] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:05 2022] DELETE /identity/v3/users/a329a3c5c135413fa2aea728deeba719 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:05.667295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.667595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.667866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.674835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da4bad51e454e378b645807bfbab8bc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.685419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da4bad51e454e378b645807bfbab8bc'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:05.690508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:05.692339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.693574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 893/1802] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:05 2022] DELETE /identity/v3/users/6da4bad51e454e378b645807bfbab8bc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:05.765898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 910/1803] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:05.786300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:05.788143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:05.901800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e56d27-b926-4aa5-a202-6316ca8df4d2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:05.916027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:05.916901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:05.917341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:05.917796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:05.949072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:05.951664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62569e40-a0d1-4d4a-90fc-80f0f43fc46b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:05.971422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 894/1804] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:06.120735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.122067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.149092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f383161-3a81-43d9-8fa3-a69a5ee72c31 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.165997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.167980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.168272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.168615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] PATH_INFO: `/v3/projects/77652b6c6ae640dca95f8908c408ed6c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.170174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77652b6c6ae640dca95f8908c408ed6c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.173077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.231168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:06.312448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77652b6c6ae640dca95f8908c408ed6c', 'request_id': 'req-65546178-5d2b-47c2-852c-297e8415d65d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.326251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77652b6c6ae640dca95f8908c408ed6c', 'request_id': 'req-65546178-5d2b-47c2-852c-297e8415d65d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.335352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77652b6c6ae640dca95f8908c408ed6c', 'request_id': 'req-65546178-5d2b-47c2-852c-297e8415d65d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.349409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] The token cache is being invalidate because project 77652b6c6ae640dca95f8908c408ed6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:06.349667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.350325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-65546178-5d2b-47c2-852c-297e8415d65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.352434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 911/1805] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:06 2022] DELETE /identity/v3/projects/77652b6c6ae640dca95f8908c408ed6c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:06.357161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c09756-c626-475e-a7fb-11a57995e2cc None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.357503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c09756-c626-475e-a7fb-11a57995e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.357790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c09756-c626-475e-a7fb-11a57995e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.382028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6ccd-5197-4250-b6a5-d7618d7d5185 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.382028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6ccd-5197-4250-b6a5-d7618d7d5185 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.382028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6ccd-5197-4250-b6a5-d7618d7d5185 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.382540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 912/1806] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 09:12:06.395409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-29c09756-c626-475e-a7fb-11a57995e2cc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:06.395588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.395795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.396080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.398308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-29c09756-c626-475e-a7fb-11a57995e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.426398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:06.433558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:06.437516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.517757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 895/1807] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:06.525168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.525813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.526137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.561201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 913/1808] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:06.563606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:06.565167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.570857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.573210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.592796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26e93-62c6-4312-8dbc-a1a068eea4fc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.594019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 896/1809] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:06.598908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.607752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.608624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.608863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.609137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] PATH_INFO: `/v3/projects/47a116cb8061487c88930ad3db813b6d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.611453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a116cb8061487c88930ad3db813b6d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.614385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.617307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5590c4-8f9b-44da-a0b9-5eaf7590dd8c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.623374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.624420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.624997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.625453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.626768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:06.627423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.630002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.654782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 897/1810] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:06.658981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.670910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ea3c4-1c1f-45eb-8d1d-1aaefa70a947 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.677142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.678636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.679509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.679975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.681592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.685417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.714101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a116cb8061487c88930ad3db813b6d', 'request_id': 'req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.721152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 898/1811] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:06 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:06.724200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a116cb8061487c88930ad3db813b6d', 'request_id': 'req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.730119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.732195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a116cb8061487c88930ad3db813b6d', 'request_id': 'req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.759008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] The token cache is being invalidate because project 47a116cb8061487c88930ad3db813b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:06.759318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.759996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.761543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 914/1812] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:06 2022] DELETE /identity/v3/projects/47a116cb8061487c88930ad3db813b6d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:06.764432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.764432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.787178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:06.816607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b77fd-c21c-4a05-8521-b5ede798f709 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.824539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.824539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.824539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.824539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.824956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.826946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.851536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 899/1813] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:06 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:06.852624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.858414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.858705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1a86d-e3ec-4a8c-9871-ca71fd7d86a5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.871526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092685-5010-47b4-92a9-450280952014 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:06.878221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.878942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.879242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.879512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] PATH_INFO: `/v3/users/b6c1f8b8286a4c80a71daac1ead0bea8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.880713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c1f8b8286a4c80a71daac1ead0bea8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.882710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:06.942719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c1f8b8286a4c80a71daac1ead0bea8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.943780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c1f8b8286a4c80a71daac1ead0bea8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.953002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:06.959107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c1f8b8286a4c80a71daac1ead0bea8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.968340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c1f8b8286a4c80a71daac1ead0bea8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.971670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:06.973007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:06.973426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:06.973774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:06.975867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.976806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c1f8b8286a4c80a71daac1ead0bea8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:06.986060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 900/1814] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:06 2022] DELETE /identity/v3/users/b6c1f8b8286a4c80a71daac1ead0bea8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:06.986919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:06.989336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:06.994011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:06.996312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.010947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 915/1815] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:07.071242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27863ffe-67e0-4b6b-ba18-0af11ea9e63d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.078796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.079439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.079719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.079994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.080699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.084735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.099208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 901/1816] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:07.100561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.103155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.117397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30216-4003-40e8-a7c0-128fc2178c47 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.123049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.123797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.124061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.124363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.125154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.126937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd22f6-7071-493a-b1e8-410929404b4c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.141864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 916/1817] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:07.143108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.155154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9148f2-3a45-4a8b-a61d-4d8f57eb60a4 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.161157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.161914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.162183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.162461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] PATH_INFO: `/v3/projects/d6f986b9e5fc433ab9c8f954b3f5fb30/users/57e52fc3adac4a5c9a925c1aa535a4e3/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.184132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f986b9e5fc433ab9c8f954b3f5fb30, user_id=57e52fc3adac4a5c9a925c1aa535a4e3, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.186885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.205158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 902/1818] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:07 2022] PUT /identity/v3/projects/d6f986b9e5fc433ab9c8f954b3f5fb30/users/57e52fc3adac4a5c9a925c1aa535a4e3/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:07.218106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.219035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.219570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.250082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] MFA Rules not processed for user `57e52fc3adac4a5c9a925c1aa535a4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:07.251401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.310659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.312427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.319187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 917/1819] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:07.327075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.360027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.391981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba72bf-0a52-44fc-837c-675a81dd7540 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.397306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.397979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.398262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.398535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] PATH_INFO: `/v3/projects/fdf8a07539c94726bd5e5d4b846e479e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.399694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf8a07539c94726bd5e5d4b846e479e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.400873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287b74-13d3-404b-9cdf-3c25f3dcf87b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.401574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.405705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.406436 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.406797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.407139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.407874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.409822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.437882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 918/1820] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:07.441878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.448817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf8a07539c94726bd5e5d4b846e479e', 'request_id': 'req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:07.450710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.457952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf8a07539c94726bd5e5d4b846e479e', 'request_id': 'req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:07.465594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf8a07539c94726bd5e5d4b846e479e', 'request_id': 'req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:07.475970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] The token cache is being invalidate because project fdf8a07539c94726bd5e5d4b846e479e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:07.476283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:07.476897 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b8ee6dbd-9cde-4e24-9d09-221dd303d7f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:07.478724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 903/1821] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:07 2022] DELETE /identity/v3/projects/fdf8a07539c94726bd5e5d4b846e479e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:07.483229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4a40-8778-4f73-babb-dcb5b7d99207 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.483537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4a40-8778-4f73-babb-dcb5b7d99207 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.483815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4a40-8778-4f73-babb-dcb5b7d99207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.504299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-224f4a40-8778-4f73-babb-dcb5b7d99207 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:07.505216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b43ffc-0ef1-4d0b-95c1-d115b39a6a4c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.505768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-224f4a40-8778-4f73-babb-dcb5b7d99207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.510998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.511618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.511895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.512169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.512837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.514838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.515904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 904/1822] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:07.525076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.525537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.526061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.539783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 919/1823] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:07.544285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.546089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.547876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:07.549032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.557432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 905/1824] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:07.558722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321100dc-110a-4dd4-8b3d-3d7f4c0e98fc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.561664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.563295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.563906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.564184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.564455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.565143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.567320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.571543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823ed52-cc92-42f5-9aab-dfbfafbccb8e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.576863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.577458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.577560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 920/1825] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:07.577736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.578044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.578792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:07.579093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.581075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.581427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.591009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bad8-8773-4917-b9b1-8a4047ababc5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.592801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 906/1826] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:07.597255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.597479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.598189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.598472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.598747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] PATH_INFO: `/v3/projects/ad14cc668b484e428c24e808154a5984/users/ee2933780a92408b8c274e590706f963/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.606771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2100d-cee6-4e9b-b2c8-f73179cf4d1f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.611402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.611935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.612214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.612487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.613668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.614752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad14cc668b484e428c24e808154a5984, user_id=ee2933780a92408b8c274e590706f963, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.615574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.617480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.632918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 921/1827] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:07 2022] PUT /identity/v3/projects/ad14cc668b484e428c24e808154a5984/users/ee2933780a92408b8c274e590706f963/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:07.637059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.646931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 907/1828] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:07.650857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.699268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcfc5-179a-49f8-aa55-6708de4cd436 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.704108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.712442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711660e-9a91-4f4e-bb7c-72060516e9d8 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.717751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.718411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.718691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.718971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.719625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.721580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.744546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 908/1829] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:07.748659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.760415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b293-b189-4e71-bec2-f5270bb26f4e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.765610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.765714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.766059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.766229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.766329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.766481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.766569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.766763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.767221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.767514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.768873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.769291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.781392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 922/1830] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:07.785280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.790601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 909/1831] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:07.791682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.797283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c71ea-c9d9-4ae6-a7d9-b1d2fe99ac1d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.804288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.805074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.805408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.805695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] PATH_INFO: `/v3/projects/ad14cc668b484e428c24e808154a5984/users/ee2933780a92408b8c274e590706f963/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.806585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bac10-1574-4bf4-b6a6-7d8c0c0d27ac admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.807852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad14cc668b484e428c24e808154a5984, user_id=ee2933780a92408b8c274e590706f963, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.810851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2433ad-f2b7-4d0a-86a1-f835a6578e5b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.812420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.813056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.813574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.813870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.814936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.817819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.825880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 923/1832] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:07 2022] PUT /identity/v3/projects/ad14cc668b484e428c24e808154a5984/users/ee2933780a92408b8c274e590706f963/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:07.833875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 910/1833] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:07.835151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.835780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.836180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.836565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.846153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7947f-8c5f-4507-a09e-86b67e51f864 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:07.872948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] MFA Rules not processed for user `ee2933780a92408b8c274e590706f963`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:07.873678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:07.907607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:07.920732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:07.921420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.921698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.922021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] PATH_INFO: `/v3/projects/b0b30726917e4d24b57b36e689a842fb/users/81b1af786c9b47b38434ff7cf7e9515f/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.942166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b30726917e4d24b57b36e689a842fb, user_id=81b1af786c9b47b38434ff7cf7e9515f, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:07.945362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7345d7-2daa-48bc-acf1-633b5bd390c9 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:07.960555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 911/1834] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:07 2022] PUT /identity/v3/projects/b0b30726917e4d24b57b36e689a842fb/users/81b1af786c9b47b38434ff7cf7e9515f/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:07.971419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:07.971909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 924/1835] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:07.972030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:07.977480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:07.984139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:07.985546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.008753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] MFA Rules not processed for user `81b1af786c9b47b38434ff7cf7e9515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:08.010119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.050384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2c54c-40f3-4923-a1f9-065f4615dd59 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.078169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 912/1836] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:08.090321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.092029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.108949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.133829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.146054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.146054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.146054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.165710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19927-8d25-487f-9849-ff30d0925bd7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.169814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.170565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.170852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.171115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.180531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.182066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.191752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 913/1837] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:08.214082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.214082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.214256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 925/1838] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:08.215132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.284674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f68d-d207-421d-814f-be05e76122f3 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.290699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.291373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.291651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.291926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] PATH_INFO: `/v3/users/81b1af786c9b47b38434ff7cf7e9515f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.293139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b1af786c9b47b38434ff7cf7e9515f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.295101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.308121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.310407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.318443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9ba1-63c6-4d47-af68-f65833d4eec4 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.322793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.323525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.323838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.324177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.326286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.351523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b1af786c9b47b38434ff7cf7e9515f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.352511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b1af786c9b47b38434ff7cf7e9515f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.366629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b1af786c9b47b38434ff7cf7e9515f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.378504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b1af786c9b47b38434ff7cf7e9515f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.392386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-776c4090-835c-4b22-9f7c-dd774f5244aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b1af786c9b47b38434ff7cf7e9515f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.399580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 914/1839] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:08 2022] DELETE /identity/v3/users/81b1af786c9b47b38434ff7cf7e9515f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:08.402484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.402672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.403956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.404378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.404895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.412285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 926/1840] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:08.426441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:08.428045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.492992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 915/1841] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:08.502846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.504972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.569296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a2fc4-ed21-47dc-9a54-0a6ee1ceb632 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.603515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.624036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.624670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.624953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.625229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.635864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.637533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3b590-57c3-41d6-be00-415dff438ada service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.647448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 927/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:08.688335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.689724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.702993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e276-ea31-4428-bc79-48f88d0e262b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.708677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.709715 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.710084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.710369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] PATH_INFO: `/v3/users/57e52fc3adac4a5c9a925c1aa535a4e3` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.711887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e52fc3adac4a5c9a925c1aa535a4e3)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.714896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.768785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e52fc3adac4a5c9a925c1aa535a4e3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.769439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e52fc3adac4a5c9a925c1aa535a4e3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.781627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e52fc3adac4a5c9a925c1aa535a4e3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.790687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e52fc3adac4a5c9a925c1aa535a4e3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.798467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e52fc3adac4a5c9a925c1aa535a4e3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:08.805644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 916/1843] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:08 2022] DELETE /identity/v3/users/57e52fc3adac4a5c9a925c1aa535a4e3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:08.810375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.810729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.811067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.831014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:08.832617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.899920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 928/1844] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:08.910784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.912463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.920175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:08.936340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:08.937997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:08.953677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f234fa9-bd14-42f8-abda-efa3db2a1d4e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.960422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.961168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.961458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.961865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] PATH_INFO: `/v3/projects/b0b30726917e4d24b57b36e689a842fb` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:08.963108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b30726917e4d24b57b36e689a842fb)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:08.965046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:08.985244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb48da-2aea-4e01-b4ca-e053437c30eb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:08.989682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:08.990408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:08.990767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:08.991112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.000488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.001968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7eae51-4f16-489e-9221-8fab86d6479a service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.012681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 917/1845] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:09.015720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b30726917e4d24b57b36e689a842fb', 'request_id': 'req-66edeb37-d683-42f3-a715-29794d0d42ec'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.024807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b30726917e4d24b57b36e689a842fb', 'request_id': 'req-66edeb37-d683-42f3-a715-29794d0d42ec'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.031849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b30726917e4d24b57b36e689a842fb', 'request_id': 'req-66edeb37-d683-42f3-a715-29794d0d42ec'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.044976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] The token cache is being invalidate because project b0b30726917e4d24b57b36e689a842fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:09.045270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.046011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-66edeb37-d683-42f3-a715-29794d0d42ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.048324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 929/1846] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:08 2022] DELETE /identity/v3/projects/b0b30726917e4d24b57b36e689a842fb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:09.054695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.055194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.055412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.082045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:09.083319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.146096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 918/1847] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:09.155086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.155303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.155620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.177157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:09.178443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.187923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 930/1848] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:09.192921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.203711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dad0c-cc02-4893-825d-2d9f457e0139 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.208186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.208815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.209102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.209382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.210124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:09.210435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.212106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.224659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 919/1849] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:09.229166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.239189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ffd5f-83b0-4ef7-abf1-a2a67e25c086 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.244381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.245108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.245461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.245806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.246569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.248560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.280676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 931/1850] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:09.281870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.292697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c467e88-49d0-4203-94e3-e2fcc40384fe admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.375497 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.376138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.376428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.376688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.378154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.380807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.420706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.422732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 920/1851] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:09 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:09.432048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a37fee-cc81-45e9-8e41-44aaff047d24 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.438596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.439439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.439887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.440164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.440920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.442983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.474641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 932/1852] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:09.478626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.493526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202033b-03d5-44d5-9cad-69b82f1039ae admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.494702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.497172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.509614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.510841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.511291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.511881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.512576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44dce0-1ada-4274-a747-88f8b555e9ee admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.513033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.514947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.523544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.524210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.524490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.524764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] PATH_INFO: `/v3/projects/d6f986b9e5fc433ab9c8f954b3f5fb30` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.526034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f986b9e5fc433ab9c8f954b3f5fb30)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.528005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.529602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 921/1853] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:09.538888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.549584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed58fe-d6da-4d1f-8235-847eb955a81e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.552139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.555588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.556345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.556720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.557114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] PATH_INFO: `/v3/projects/56f68c5de97d44c78a62bf47267545a9/users/ef3eab78e93f4c599e3c54dde143c2b1/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.569134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.577359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f68c5de97d44c78a62bf47267545a9, user_id=ef3eab78e93f4c599e3c54dde143c2b1, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.580562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32ec81-2da4-408c-bd07-5da92ffce684 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.587942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f986b9e5fc433ab9c8f954b3f5fb30', 'request_id': 'req-ff553af7-d21c-4a7f-be7b-510612db5d1e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.597708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f986b9e5fc433ab9c8f954b3f5fb30', 'request_id': 'req-ff553af7-d21c-4a7f-be7b-510612db5d1e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.601078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 922/1854] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:09 2022] PUT /identity/v3/projects/56f68c5de97d44c78a62bf47267545a9/users/ef3eab78e93f4c599e3c54dde143c2b1/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:09.605077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f986b9e5fc433ab9c8f954b3f5fb30', 'request_id': 'req-ff553af7-d21c-4a7f-be7b-510612db5d1e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.611427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.611719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.612583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.618121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] The token cache is being invalidate because project d6f986b9e5fc433ab9c8f954b3f5fb30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:09.618567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.619411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.621373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 933/1855] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:09 2022] DELETE /identity/v3/projects/d6f986b9e5fc433ab9c8f954b3f5fb30 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:09.626747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.628094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.651948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] MFA Rules not processed for user `ef3eab78e93f4c599e3c54dde143c2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:09.653182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.710928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff553af7-d21c-4a7f-be7b-510612db5d1e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.728588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 923/1856] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:09.735981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:09.737355 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:09.768889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.770289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.770632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.770977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] PATH_INFO: `/v3/users/ee2933780a92408b8c274e590706f963` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.772205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2933780a92408b8c274e590706f963)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.774226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.793758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37434618-c14c-409d-88fa-c5cf3a70b24c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:09.797446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:09.798105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:09.798393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:09.798671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:09.807391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:09.808826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:09.817753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 924/1857] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:09.820054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2933780a92408b8c274e590706f963'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.828110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:09.836146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2933780a92408b8c274e590706f963'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.843814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2933780a92408b8c274e590706f963'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.844540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2933780a92408b8c274e590706f963'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.845157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-03da525f-e268-4c77-acf5-3ff7c421e10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2933780a92408b8c274e590706f963'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:09.858305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 934/1858] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:09 2022] DELETE /identity/v3/users/ee2933780a92408b8c274e590706f963 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:10.155992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.169338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dac136-2bda-44bb-857a-1af20a3ec0f6 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.228546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.229209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.229488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.229768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] PATH_INFO: `/v3/projects/ad14cc668b484e428c24e808154a5984` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.231172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad14cc668b484e428c24e808154a5984)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.233450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.258643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:10.281410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad14cc668b484e428c24e808154a5984', 'request_id': 'req-4f7f0df6-9136-4f95-9179-73a2e03de93a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:10.289209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad14cc668b484e428c24e808154a5984', 'request_id': 'req-4f7f0df6-9136-4f95-9179-73a2e03de93a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:10.295545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad14cc668b484e428c24e808154a5984', 'request_id': 'req-4f7f0df6-9136-4f95-9179-73a2e03de93a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:10.306334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] The token cache is being invalidate because project ad14cc668b484e428c24e808154a5984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:10.306624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:10.307264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4f7f0df6-9136-4f95-9179-73a2e03de93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:10.308782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 925/1859] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:10 2022] DELETE /identity/v3/projects/ad14cc668b484e428c24e808154a5984 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:10.313638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.314067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.314423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.335335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:10.336545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:10.400565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 935/1860] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:10.408272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.408621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.408833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.430867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:10.431268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:10.442797 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 926/1861] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:10.443968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.456125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1137248-4584-4026-8944-aa12f3db0893 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.461509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.462100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.462387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.462661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.463360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:10.463667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.465824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.478467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 936/1862] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:10.482340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.491787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0b5ee-a5bf-43dd-a27a-42f4dcfa2c3c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.496628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.497253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.497531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.497805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.498553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.500394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.532324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 927/1863] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:10 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:10.533980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.544605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd7a-e06c-4ad8-80a9-4b16c0fef579 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.622732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.623380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.623647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.623925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.624608 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.626710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.652812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 937/1864] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:10 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:10.656937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.668725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b8e1f-7273-43fa-a4c2-e1d232813a3b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.673995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.674625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.674904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.675170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.675911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.677517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.689289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 928/1865] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:10.701861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.712512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7706f-92a5-4d0d-99a6-a4c82d852839 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.717540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.718419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.718804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.719152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.719942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.721674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838c5ff-829a-43cf-9159-9e46bf0ba96c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.737148 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 938/1866] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:10.743752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.753971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6795-fb5b-4e6d-bdf0-4fdd3b209f63 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.758551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.759358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.759479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.759753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] PATH_INFO: `/v3/projects/bb12343d6f014f9895125e19d02b4849/users/7fb19714f22c490cbacfa339cb44570b/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.774985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb12343d6f014f9895125e19d02b4849, user_id=7fb19714f22c490cbacfa339cb44570b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.777635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.793040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 929/1867] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:10 2022] PUT /identity/v3/projects/bb12343d6f014f9895125e19d02b4849/users/7fb19714f22c490cbacfa339cb44570b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:10.798165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3dae-ee29-47e8-958b-9d8b2484d9cf None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.798525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3dae-ee29-47e8-958b-9d8b2484d9cf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.798801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3dae-ee29-47e8-958b-9d8b2484d9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.817535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-911f3dae-ee29-47e8-958b-9d8b2484d9cf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:10.819085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-911f3dae-ee29-47e8-958b-9d8b2484d9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:10.857316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:10.886972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 939/1868] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:10.898183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:10.900504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:10.964789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ae4e-a899-483d-bb86-d6f38d6ef937 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:10.968369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:10.968988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:10.969270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:10.969545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:10.978752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:10.980207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:10.989460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 930/1869] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:12.642681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476116ba-8991-4134-b328-a224b059283b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.643196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476116ba-8991-4134-b328-a224b059283b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.643275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476116ba-8991-4134-b328-a224b059283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.681156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-476116ba-8991-4134-b328-a224b059283b None None] MFA Rules not processed for user `7fb19714f22c490cbacfa339cb44570b`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:12.682697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-476116ba-8991-4134-b328-a224b059283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:12.774451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 940/1870] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:12.779103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.788937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec74d-0b49-4ad1-be19-a64312b02f7b service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.794136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.794979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.794979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.795256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.795927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.798142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:12.830914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 931/1871] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:12 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:12.835691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476116ba-8991-4134-b328-a224b059283b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.846874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476116ba-8991-4134-b328-a224b059283b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.916870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.917661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.918061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.918392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.919144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.921195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:12.952472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 941/1872] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:12 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:12.957553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:12.969220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3b6b6-367f-40f0-9a04-82bd028f146a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:12.974325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:12.975021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:12.975777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:12.975777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:12.976498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:12.978555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:12.993833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 932/1873] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:12.995956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.007604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc134-6099-4f14-94ba-7c0170a1055b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.012479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.013639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.013639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.013872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] PATH_INFO: `/v3/projects/e2ee154316de4a72b654f83030a9a1fc/users/44f4385235b945528aff09549c237d3b/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.036216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ee154316de4a72b654f83030a9a1fc, user_id=44f4385235b945528aff09549c237d3b, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.039639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.057384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 942/1874] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:12 2022] PUT /identity/v3/projects/e2ee154316de4a72b654f83030a9a1fc/users/44f4385235b945528aff09549c237d3b/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:13.061866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.075799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67783d0-a148-4d25-8ac4-7be40cf2074e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.095234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:13.196408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.199563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.199928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.200300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.203014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.206970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967018dc-e2e8-4c18-a77c-4bad74525010 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.226726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 933/1875] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:13.228093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:13.239450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed3183-d84a-4ece-9dff-577c8c2a1a59 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:13.244836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:13.245519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:13.245796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:13.246122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] PATH_INFO: `/v3/projects/e2ee154316de4a72b654f83030a9a1fc/users/44f4385235b945528aff09549c237d3b/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:13.248107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ee154316de4a72b654f83030a9a1fc, user_id=44f4385235b945528aff09549c237d3b, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:13.250811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:13.265742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 943/1876] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:13 2022] PUT /identity/v3/projects/e2ee154316de4a72b654f83030a9a1fc/users/44f4385235b945528aff09549c237d3b/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:14.380855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.381363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.381464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.410744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] MFA Rules not processed for user `44f4385235b945528aff09549c237d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:14.411963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:14.488867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 934/1877] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:14.496686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:14.501899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:14.528315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:14.562288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13211fae-3be9-41ec-96f2-432b1f56a24b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:14.565944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:14.566571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:14.566853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:14.567130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:14.627293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:14.628973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:14.639524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 944/1878] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:17.071448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:17.073464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:17.086049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be510a-c5b4-4eb7-af5a-a31185d3fae3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:17.089594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:17.090239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:17.090518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:17.090789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:17.099717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:17.101410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5088-9af8-42d1-b1a0-8b7f63d72c81 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:17.110801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 935/1879] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:23.212900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.214491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:23.289723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc369-ffe2-4df8-b5b0-daca7ab7c32d service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.294252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.294894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.295170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.295438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] PATH_INFO: `/v3/users/4fb1f69572964cffae3bd9aa19e2af11` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.296609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb1f69572964cffae3bd9aa19e2af11)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.299597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.347170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb1f69572964cffae3bd9aa19e2af11'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:23.360231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb1f69572964cffae3bd9aa19e2af11'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:23.367675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb1f69572964cffae3bd9aa19e2af11'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:23.369056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb1f69572964cffae3bd9aa19e2af11'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:23.369816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb1f69572964cffae3bd9aa19e2af11'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:23.374443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:23.388774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 945/1880] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:23 2022] DELETE /identity/v3/users/4fb1f69572964cffae3bd9aa19e2af11 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:23.393412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.394231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.395159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.439804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:23.441026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:23.543946 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 936/1881] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:23.592480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:23.594750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:23.694155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc39ca-9265-429e-9d4e-2480a6294993 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:23.699236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:23.700334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:23.700808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:23.701297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:23.713111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:23.714864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:23.737994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 946/1882] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:24.733086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.734901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:24.758484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520db4-5833-41fb-929e-999068dd1da8 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.764034 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.765059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.765417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.765767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] PATH_INFO: `/v3/projects/d962b65c517449c7ad4a9ae343273c81` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.767085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d962b65c517449c7ad4a9ae343273c81)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.769178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:24.795350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:24.820534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd962b65c517449c7ad4a9ae343273c81', 'request_id': 'req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:24.827745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd962b65c517449c7ad4a9ae343273c81', 'request_id': 'req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:24.835925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd962b65c517449c7ad4a9ae343273c81', 'request_id': 'req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:24.849276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] The token cache is being invalidate because project d962b65c517449c7ad4a9ae343273c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:24.849744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:24.850597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:24.852491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 937/1883] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:24 2022] DELETE /identity/v3/projects/d962b65c517449c7ad4a9ae343273c81 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:24.856820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:24.867371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c50f93-ab17-48bd-955c-814915b3d731 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:24.929642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:24.930330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:24.930613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:24.930889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] PATH_INFO: `/v3/users/90b3509a88ee49d88dec42183f2889b2` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:24.932061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b3509a88ee49d88dec42183f2889b2)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:24.934051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:25.014612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b3509a88ee49d88dec42183f2889b2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.038727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b3509a88ee49d88dec42183f2889b2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.046967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b3509a88ee49d88dec42183f2889b2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.047789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b3509a88ee49d88dec42183f2889b2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.048502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2f7d16cf-14a2-4060-a6a6-f5c31f85d01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b3509a88ee49d88dec42183f2889b2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.064612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 947/1884] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:24 2022] DELETE /identity/v3/users/90b3509a88ee49d88dec42183f2889b2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:25.359131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:25.360673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:25.374452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93535c-c2cd-4c4f-9e07-2a2efa0b7adf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:25.438773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:25.439426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.439707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.439981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] PATH_INFO: `/v3/projects/b23776708ea84754b0b8d24c79ffffa6` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.441291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23776708ea84754b0b8d24c79ffffa6)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:25.443894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:25.500471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23776708ea84754b0b8d24c79ffffa6', 'request_id': 'req-c6b78d9e-8e06-493c-a2bd-843430934661'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.510411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23776708ea84754b0b8d24c79ffffa6', 'request_id': 'req-c6b78d9e-8e06-493c-a2bd-843430934661'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.517676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23776708ea84754b0b8d24c79ffffa6', 'request_id': 'req-c6b78d9e-8e06-493c-a2bd-843430934661'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.531396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] The token cache is being invalidate because project b23776708ea84754b0b8d24c79ffffa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:25.531688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.532512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6b78d9e-8e06-493c-a2bd-843430934661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:25.534438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 938/1885] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:25 2022] DELETE /identity/v3/projects/b23776708ea84754b0b8d24c79ffffa6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:25.539832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.540244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.540669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.561495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:25.562763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:25.578948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:25.703999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 948/1886] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:25.706872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473add-2312-47eb-96d0-16b67646430e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.706872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473add-2312-47eb-96d0-16b67646430e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.706872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473add-2312-47eb-96d0-16b67646430e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.756620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-be473add-2312-47eb-96d0-16b67646430e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:25.759200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-be473add-2312-47eb-96d0-16b67646430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:25.769160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 939/1887] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:25.774182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:25.787446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90444-9386-4bf9-93aa-4f7ccf2ec1ef None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:25.793441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:25.794238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.794636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.795158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.796191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:25.796591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:25.798915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:25.845606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 949/1888] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:25.850600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473add-2312-47eb-96d0-16b67646430e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:25.861710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473add-2312-47eb-96d0-16b67646430e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:25.888416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:25.889485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:25.889885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:25.891267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:25.891267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:25.893230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:25.936925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 940/1889] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:25 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:25.941197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:25.957728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55848476-c4d9-42b3-9100-5b3590cf3f1b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.040719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.041675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.042171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.042468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.043567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.045543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.070232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 950/1890] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:25 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:26.074666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.085568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc216d-ff9c-4709-8474-f25802a9396f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.091231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.091883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.092166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.092444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.093149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.094816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.103051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:26.109822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 941/1891] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:26.115011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.126531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3626e0-1c3d-4f93-a6e2-ef4f67b51487 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.131628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.132679 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.133067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.133358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.134170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.135868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e365cc-1cc1-4171-a9f5-2837bc8c677e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.162754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 951/1892] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:26.166567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.177393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9791e0-5469-4c44-8691-10fc30048742 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.182173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.182913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.183286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.183573 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] PATH_INFO: `/v3/projects/827c678b6a8e438a90f8617d81e5524e/users/bc05a44a34514cb0bb7cc94c6792a4e5/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.201426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=827c678b6a8e438a90f8617d81e5524e, user_id=bc05a44a34514cb0bb7cc94c6792a4e5, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.204188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.223694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 942/1893] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:26 2022] PUT /identity/v3/projects/827c678b6a8e438a90f8617d81e5524e/users/bc05a44a34514cb0bb7cc94c6792a4e5/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:26.235553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.235742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.236018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.268511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] MFA Rules not processed for user `bc05a44a34514cb0bb7cc94c6792a4e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:26.270720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:26.386169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 952/1894] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:26.388566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.451047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0fb70-ac43-40a9-bb6c-a1b3a1243cdf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.456212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.456835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.457153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.457426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.458584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.460053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.492068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 943/1895] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:26 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:26.492767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.515223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed193e-aa8f-4321-901f-5361a383cf81 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.526736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:26.581379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.582087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.582373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.582651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.583329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.585251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.607289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 953/1896] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:26 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:26.617263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.628689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5813-32ee-4b39-b4b1-8b16aec3ce23 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.634264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.634883 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.635168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.635447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.636144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.637755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.648350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 944/1897] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:26.652535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.662984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c1e0-bfe1-4f0f-804e-f693a425c61c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.667568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.668226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.668509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.668785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] PATH_INFO: `/v3/projects/a8e337618ce7474eb0178dd9f064b2cf/users/fad4cfe9bb7d4255a1303d0dea4005df/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.683742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e337618ce7474eb0178dd9f064b2cf, user_id=fad4cfe9bb7d4255a1303d0dea4005df, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.686445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.700056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 954/1898] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:26 2022] PUT /identity/v3/projects/a8e337618ce7474eb0178dd9f064b2cf/users/fad4cfe9bb7d4255a1303d0dea4005df/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:26.703500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.714014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269cc9-a0b8-44e6-b390-e52cc58e0063 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.774502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.775152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.775320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.775600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.776332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.777964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86630e-cc26-4e59-818e-dc92626c91cf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.788402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 945/1899] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:26.792685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:26.804146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5382e1-2510-4f20-8ec4-c43ee0bc3c47 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:26.809133 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:26.809820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.810214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.810488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] PATH_INFO: `/v3/projects/a8e337618ce7474eb0178dd9f064b2cf/users/fad4cfe9bb7d4255a1303d0dea4005df/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.813303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e337618ce7474eb0178dd9f064b2cf, user_id=fad4cfe9bb7d4255a1303d0dea4005df, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:26.818630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:26.830578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 955/1900] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:26 2022] PUT /identity/v3/projects/a8e337618ce7474eb0178dd9f064b2cf/users/fad4cfe9bb7d4255a1303d0dea4005df/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:26.842119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:26.842380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:26.842669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:26.871140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] MFA Rules not processed for user `fad4cfe9bb7d4255a1303d0dea4005df`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:26.872367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:26.920396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:26.967260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 946/1901] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:27.001381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.003291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:27.063537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d992dea-a4dd-4a17-ae28-ce726a0dcd04 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.067909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.068539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.068867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.069084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.083457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:27.085098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed2972-1af2-4946-9965-5920cd2b9f15 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:27.095993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 956/1902] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:27.525371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.598617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772c42e-c65e-4ffa-ba37-3306f2b920ea None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.603212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.603865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.604150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.604429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] PATH_INFO: `/v3/users/bc05a44a34514cb0bb7cc94c6792a4e5` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.606411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc05a44a34514cb0bb7cc94c6792a4e5)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:27.608360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:27.668070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc05a44a34514cb0bb7cc94c6792a4e5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:27.668734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc05a44a34514cb0bb7cc94c6792a4e5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:27.681461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc05a44a34514cb0bb7cc94c6792a4e5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:27.689641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc05a44a34514cb0bb7cc94c6792a4e5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:27.698401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc05a44a34514cb0bb7cc94c6792a4e5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:27.705113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 947/1903] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:27 2022] DELETE /identity/v3/users/bc05a44a34514cb0bb7cc94c6792a4e5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:27.714904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.715189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.715463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.735770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:27.737008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:27.803649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 957/1904] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:27.813261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:27.814765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:27.871425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3f280-69e1-4a66-bb4e-032bf5f20eee admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:27.875003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:27.875620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:27.875904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:27.876182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:27.882532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:27.887431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:27.889238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:27.900118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 948/1905] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:28.198729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.200296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:28.212886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2a324-1a0f-4e44-b990-5571eb5d4c76 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.218280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.218951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.219237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.219531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] PATH_INFO: `/v3/projects/827c678b6a8e438a90f8617d81e5524e` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.220753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=827c678b6a8e438a90f8617d81e5524e)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.222763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.225435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:28.281166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827c678b6a8e438a90f8617d81e5524e', 'request_id': 'req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.289713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827c678b6a8e438a90f8617d81e5524e', 'request_id': 'req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.296298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827c678b6a8e438a90f8617d81e5524e', 'request_id': 'req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.307591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] The token cache is being invalidate because project 827c678b6a8e438a90f8617d81e5524e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:28.307878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.308516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.310156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 958/1906] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:28 2022] DELETE /identity/v3/projects/827c678b6a8e438a90f8617d81e5524e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:28.314019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.324491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c631-f4da-4b63-9da2-e7e399279253 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:28.377664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:28.378245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:28.378526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:28.378801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] PATH_INFO: `/v3/users/fad4cfe9bb7d4255a1303d0dea4005df` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:28.380547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad4cfe9bb7d4255a1303d0dea4005df)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:28.382491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:28.435022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad4cfe9bb7d4255a1303d0dea4005df'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.435768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad4cfe9bb7d4255a1303d0dea4005df'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.451477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad4cfe9bb7d4255a1303d0dea4005df'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.459266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad4cfe9bb7d4255a1303d0dea4005df'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.466550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-47af5b89-096c-4e07-ac72-92f4f82f26c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad4cfe9bb7d4255a1303d0dea4005df'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:28.473211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 949/1907] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:28 2022] DELETE /identity/v3/users/fad4cfe9bb7d4255a1303d0dea4005df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:28.896217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:28.897702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:28.928066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d102cf6-d761-46b8-8b8d-7a9dbbd712dc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.007446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.008237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.008676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.009043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] PATH_INFO: `/v3/projects/a8e337618ce7474eb0178dd9f064b2cf` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.010325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e337618ce7474eb0178dd9f064b2cf)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.012462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.070432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e337618ce7474eb0178dd9f064b2cf', 'request_id': 'req-771ef2f0-a332-4955-bdb5-f9c555fc501e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:29.079339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e337618ce7474eb0178dd9f064b2cf', 'request_id': 'req-771ef2f0-a332-4955-bdb5-f9c555fc501e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:29.086660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e337618ce7474eb0178dd9f064b2cf', 'request_id': 'req-771ef2f0-a332-4955-bdb5-f9c555fc501e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:29.099473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] The token cache is being invalidate because project a8e337618ce7474eb0178dd9f064b2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:29.099761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:29.100417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-771ef2f0-a332-4955-bdb5-f9c555fc501e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:29.102019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 959/1908] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:28 2022] DELETE /identity/v3/projects/a8e337618ce7474eb0178dd9f064b2cf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:29.108016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12669-5946-4a62-8853-576af8eee306 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.108569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12669-5946-4a62-8853-576af8eee306 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.108663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12669-5946-4a62-8853-576af8eee306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.130422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bea12669-5946-4a62-8853-576af8eee306 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:29.131732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bea12669-5946-4a62-8853-576af8eee306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:29.199984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 950/1909] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:29.208283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.208796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.208796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.234035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:29.234035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:29.243292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 960/1910] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:29.247779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12669-5946-4a62-8853-576af8eee306 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.258132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12669-5946-4a62-8853-576af8eee306 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.262814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.263587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.263679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.263945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.264640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:29.264942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.266561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.277615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:29.282154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 951/1911] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:29.286791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.296475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594a5b9-8d5d-4228-b8eb-3e15072979c5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.302694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.303437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.303726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.304028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.304701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.306701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.327183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:29.346637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 961/1912] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:29.351064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.361473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db769438-6924-460f-ab48-bf4661695677 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.416749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.417284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.417563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.417857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.418522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.420426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.441911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 952/1913] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:29.446085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.456451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa89d0-8a00-44e9-8c9a-365bfa769e3c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.461033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.461646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.462041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.462319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.463046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.464642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.474978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 962/1914] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:29.478824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.489099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452e23-3ea0-4d2e-a146-761b8da5d6b0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.493598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.494252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.494532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.494806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.495486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.497201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357ba9-9a75-41d2-a7a4-25b6a2cadf30 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.507278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 953/1915] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:29.511170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.521060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e913c-b5a6-4b89-bdda-8c053952b7c2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.525696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.526390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.526675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.526952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] PATH_INFO: `/v3/projects/1743529332dc43888d6132d93e921ad9/users/6b0295c75da44612803cbe5a42d968b4/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.541771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1743529332dc43888d6132d93e921ad9, user_id=6b0295c75da44612803cbe5a42d968b4, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.544449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.558550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 963/1916] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:29 2022] PUT /identity/v3/projects/1743529332dc43888d6132d93e921ad9/users/6b0295c75da44612803cbe5a42d968b4/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:29.569816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.570135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.570408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.598869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] MFA Rules not processed for user `6b0295c75da44612803cbe5a42d968b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:29.600050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:29.663864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 954/1917] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:29.668447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.739778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb605a-01f6-43e7-ac74-77dd9eee52c8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.744559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.745190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.745667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.745792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.746439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.748359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.777165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 964/1918] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:29.781440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.793324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd11eaf-37ca-4d07-b59b-1190fc32292c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.851346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.851973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.852249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.852516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.853188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.855150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.872993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:29.881893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 955/1919] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:29 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:29.885504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.897098 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86072a-9e44-47db-9fd1-a3b493b9e00f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.902164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.902982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.903343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.903693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.904811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.906547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.920209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 965/1920] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:29.924118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.938218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda99e74-c8a3-4067-9179-327d19beab9f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:29.943468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:29.944135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:29.944417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:29.944689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] PATH_INFO: `/v3/projects/21f66be23c7140f28a028acc58c15d5c/users/da1a1f7a5dc84da99ae17bd384c74951/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:29.960072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f66be23c7140f28a028acc58c15d5c, user_id=da1a1f7a5dc84da99ae17bd384c74951, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:29.962746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:29.976246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 956/1921] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:29 2022] PUT /identity/v3/projects/21f66be23c7140f28a028acc58c15d5c/users/da1a1f7a5dc84da99ae17bd384c74951/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:29.980200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:29.990847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0730231-fb02-43fc-b589-d2477d451f5a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:30.052165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:30.052798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:30.053081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:30.053357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:30.054086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:30.055688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9656fb-b0d7-40ff-bfa3-434d2d583010 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:30.066462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 966/1922] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:30.070254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:30.080618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b78a1-8abf-4a13-a853-7c742a212bc5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:30.085644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:30.086345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:30.086634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:30.086909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] PATH_INFO: `/v3/projects/21f66be23c7140f28a028acc58c15d5c/users/da1a1f7a5dc84da99ae17bd384c74951/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:30.088839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f66be23c7140f28a028acc58c15d5c, user_id=da1a1f7a5dc84da99ae17bd384c74951, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:30.091486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:30.102923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 957/1923] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:30 2022] PUT /identity/v3/projects/21f66be23c7140f28a028acc58c15d5c/users/da1a1f7a5dc84da99ae17bd384c74951/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:30.112827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:30.113119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:30.114099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:30.141518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] MFA Rules not processed for user `da1a1f7a5dc84da99ae17bd384c74951`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:30.142812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:30.159635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:30.224265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 967/1924] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:30.236288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:30.238049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:30.301582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb71e4-b92a-460c-8273-9ea41ea2d120 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:30.305290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:30.305943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:30.306228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:30.306507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:30.315682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:30.317152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba7e39-2022-4253-9011-164695909477 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:30.326994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 958/1925] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:31.773277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:31.834692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828e4b0-3442-40c0-98fa-d6cb3cd9e2b9 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:31.839555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:31.840216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:31.840493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:31.840762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] PATH_INFO: `/v3/users/6b0295c75da44612803cbe5a42d968b4` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:31.842173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0295c75da44612803cbe5a42d968b4)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:31.844503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:31.888481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0295c75da44612803cbe5a42d968b4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:31.900704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0295c75da44612803cbe5a42d968b4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:31.909044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0295c75da44612803cbe5a42d968b4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:31.909757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0295c75da44612803cbe5a42d968b4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:31.910402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0295c75da44612803cbe5a42d968b4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:31.929636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 968/1926] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:31 2022] DELETE /identity/v3/users/6b0295c75da44612803cbe5a42d968b4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:31.933146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:31.933622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:31.933740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:31.953429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:31.954706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:32.011633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 959/1927] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:32.021216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:32.022664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:32.058984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:32.084347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a863c0-5944-4976-9804-e74a16f5f6d6 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.088060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.088676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.088952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.089220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.098038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.099497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.109091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 969/1928] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:32.561302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:32.562866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:32.586007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef4f4e-137a-493e-b1a9-56e5c7917430 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.597059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.598220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.599510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.599988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] PATH_INFO: `/v3/projects/1743529332dc43888d6132d93e921ad9` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.601668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1743529332dc43888d6132d93e921ad9)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.604031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.683934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1743529332dc43888d6132d93e921ad9', 'request_id': 'req-4897ddae-2985-44eb-8d5b-87e611501481'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.696076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1743529332dc43888d6132d93e921ad9', 'request_id': 'req-4897ddae-2985-44eb-8d5b-87e611501481'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.699137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:32.723339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1743529332dc43888d6132d93e921ad9', 'request_id': 'req-4897ddae-2985-44eb-8d5b-87e611501481'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.740215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] The token cache is being invalidate because project 1743529332dc43888d6132d93e921ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:32.740609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.741614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.746586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 960/1929] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:32 2022] DELETE /identity/v3/projects/1743529332dc43888d6132d93e921ad9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:32.747882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:32.773545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b4fe-4016-4b98-8175-d098a040e6f2 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:32.884318 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:32.885434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:32.885994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:32.887174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] PATH_INFO: `/v3/users/da1a1f7a5dc84da99ae17bd384c74951` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:32.889149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1a1f7a5dc84da99ae17bd384c74951)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:32.892234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:32.960186 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1a1f7a5dc84da99ae17bd384c74951'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.974314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1a1f7a5dc84da99ae17bd384c74951'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.982154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1a1f7a5dc84da99ae17bd384c74951'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.982950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1a1f7a5dc84da99ae17bd384c74951'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.983571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4980d137-c70c-4f41-bfa8-59e6832d7567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1a1f7a5dc84da99ae17bd384c74951'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:32.997515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 970/1930] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:32 2022] DELETE /identity/v3/users/da1a1f7a5dc84da99ae17bd384c74951 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:33.432437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:33.435662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:33.454669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ddae-2985-44eb-8d5b-87e611501481 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:33.560334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:33.561000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:33.561283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:33.561560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] PATH_INFO: `/v3/projects/21f66be23c7140f28a028acc58c15d5c` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:33.562871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f66be23c7140f28a028acc58c15d5c)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:33.565038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:33.667178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f66be23c7140f28a028acc58c15d5c', 'request_id': 'req-72335c68-8ed2-4f86-b5b5-858be0a4291c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:33.681930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f66be23c7140f28a028acc58c15d5c', 'request_id': 'req-72335c68-8ed2-4f86-b5b5-858be0a4291c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:33.691157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f66be23c7140f28a028acc58c15d5c', 'request_id': 'req-72335c68-8ed2-4f86-b5b5-858be0a4291c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:33.708027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] The token cache is being invalidate because project 21f66be23c7140f28a028acc58c15d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:33.708526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:33.709673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-72335c68-8ed2-4f86-b5b5-858be0a4291c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:33.712995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 961/1931] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:33 2022] DELETE /identity/v3/projects/21f66be23c7140f28a028acc58c15d5c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:33.718416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:33.718807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:33.719265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:33.759827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:33.763576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:33.839194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:33.859610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 971/1932] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:33.861652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:33.861979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:33.862263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:33.887552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:33.889875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:33.905751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 962/1933] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:33.906735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:33.931042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74452e-8e04-441f-9273-19b4348a3736 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:33.940509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:33.941249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:33.941596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:33.942556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:33.943373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:33.943775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:33.946550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:33.962298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 972/1934] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:12:33.966799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:33.978612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637bd7a-8cff-409a-a07d-fe9166749604 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:33.984824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:33.985532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:33.985922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:33.986294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:33.987814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:33.990865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.074788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 963/1935] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:33 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:34.078868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.095478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92827ca-2cec-426b-b0bd-5ec536a498f2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.195807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.196445 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.196776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.197077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.198718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.201808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.227884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 973/1936] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:34.231736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.243442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5411-80e2-4d05-8cb3-a68439797fd6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.248921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.250001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.250377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.250752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.251560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.253366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.265932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 964/1937] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:34.269169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.279832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071c735-3c4b-4c28-8e52-69fe95cf7c25 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.284624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.285331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.285676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.286058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.286760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.288372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0670a-f432-4fb8-958c-f566ce0c7854 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.299057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 974/1938] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:34.304157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.314678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725b2b-11ac-4bac-98d7-60ce64fbee74 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.320402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.321113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.321452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.321785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] PATH_INFO: `/v3/projects/0ebf2c8c9aaf4750aeee33df1cb559a4/users/0f67a8d9a7614e73bdfa0eca8497aea3/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.337812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebf2c8c9aaf4750aeee33df1cb559a4, user_id=0f67a8d9a7614e73bdfa0eca8497aea3, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.340483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.352812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 965/1939] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:34 2022] PUT /identity/v3/projects/0ebf2c8c9aaf4750aeee33df1cb559a4/users/0f67a8d9a7614e73bdfa0eca8497aea3/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:34.364430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.364720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.364996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.393960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] MFA Rules not processed for user `0f67a8d9a7614e73bdfa0eca8497aea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:34.395211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:34.465589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 975/1940] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:34.470315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.484177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:34.536944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932576-0ee7-466a-9185-e31ac116528c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.541816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.542509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.542812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.543084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.543753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.545640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.572058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 966/1941] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:34.576396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.587271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c2f3-7f35-4f5a-9d88-a6072656b218 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.635168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:34.646864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.647486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.647758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.648035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.648702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.650663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.671693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 976/1942] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:34.675618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.686768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee9ee7-80b9-4f2a-b15e-ad1d33a7ad6a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.691673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.692286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.692565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.692839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.693531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.695155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.706356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 967/1943] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:34.710260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.719903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b027652-c41a-433e-97e6-386634d56076 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.725289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.725986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.726267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.726798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] PATH_INFO: `/v3/projects/1ba87303a0364e22b32c9f442697649a/users/ab3ee8d36f78460c8415bb2c8a82e816/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.742716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba87303a0364e22b32c9f442697649a, user_id=ab3ee8d36f78460c8415bb2c8a82e816, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.745806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.762247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 977/1944] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:34 2022] PUT /identity/v3/projects/1ba87303a0364e22b32c9f442697649a/users/ab3ee8d36f78460c8415bb2c8a82e816/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:34.763261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.773405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ee5d2-db7e-4654-914a-e0c3693561c5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.833773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.834370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.834804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.835161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.835866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.837466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351b952-0d4d-43ef-a653-8018758a49b7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.847565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 968/1945] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:34.852339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:34.862972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca8ed-2bab-450d-9fca-f396dd5aceeb admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:34.867703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:34.868373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.868651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.868921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] PATH_INFO: `/v3/projects/1ba87303a0364e22b32c9f442697649a/users/ab3ee8d36f78460c8415bb2c8a82e816/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.870883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba87303a0364e22b32c9f442697649a, user_id=ab3ee8d36f78460c8415bb2c8a82e816, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:34.873590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:34.885348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 978/1946] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:34 2022] PUT /identity/v3/projects/1ba87303a0364e22b32c9f442697649a/users/ab3ee8d36f78460c8415bb2c8a82e816/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:34.896538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:34.896861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:34.897141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:34.933107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] MFA Rules not processed for user `ab3ee8d36f78460c8415bb2c8a82e816`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:34.934483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:35.002081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 969/1947] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:35.012727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:35.014460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:35.075496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c3d4-c471-4b07-81bf-d44157bcbfa9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:35.080424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:35.081104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:35.081373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:35.081644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:35.157820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:35.159371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739987b-80ba-4daf-9aeb-784cd8212816 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:35.172577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 979/1948] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:37.767661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:37.771683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:37.783699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:37.860960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9f17-31f9-4fe5-b1c2-408585393951 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:37.865662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:37.866366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:37.866646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:37.866921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] PATH_INFO: `/v3/users/ef3eab78e93f4c599e3c54dde143c2b1` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:37.868377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3eab78e93f4c599e3c54dde143c2b1)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:37.870944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:37.964190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3eab78e93f4c599e3c54dde143c2b1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:37.964873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3eab78e93f4c599e3c54dde143c2b1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:37.978886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3eab78e93f4c599e3c54dde143c2b1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:37.990290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3eab78e93f4c599e3c54dde143c2b1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.000383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3eab78e93f4c599e3c54dde143c2b1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.009380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 970/1949] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:37 2022] DELETE /identity/v3/users/ef3eab78e93f4c599e3c54dde143c2b1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:38.024966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.024966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.024966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.043771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:38.045076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:38.107974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:38.123536 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 980/1950] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:38.135173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:38.136608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:38.213433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a8bb-02ef-4db3-a930-cb5c07a4b34b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:38.217126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:38.217804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.218123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.218525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.233209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:38.234844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831272-fd9d-4202-8e06-16b3db9ab144 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:38.249775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 971/1951] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:38.628783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:38.630905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:38.648803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d1be6-dae4-4d9c-9721-0b4b30a1ab5c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:38.654361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:38.655023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.655352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.655630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] PATH_INFO: `/v3/projects/56f68c5de97d44c78a62bf47267545a9` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.657300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f68c5de97d44c78a62bf47267545a9)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:38.659330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:38.711278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f68c5de97d44c78a62bf47267545a9', 'request_id': 'req-a2485c56-3b5c-4fca-898d-261b1da9cf42'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.730425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f68c5de97d44c78a62bf47267545a9', 'request_id': 'req-a2485c56-3b5c-4fca-898d-261b1da9cf42'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.738351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f68c5de97d44c78a62bf47267545a9', 'request_id': 'req-a2485c56-3b5c-4fca-898d-261b1da9cf42'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.750881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] The token cache is being invalidate because project 56f68c5de97d44c78a62bf47267545a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:38.751251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.751972 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a2485c56-3b5c-4fca-898d-261b1da9cf42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:38.753894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 981/1952] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:38 2022] DELETE /identity/v3/projects/56f68c5de97d44c78a62bf47267545a9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:38.758754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.759044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.759615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.793506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:38.794731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:38.826944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:38.873983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 972/1953] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:12:38.877033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.877320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.877601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.897702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:38.899441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:38.912321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 982/1954] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:38.913761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:38.933652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf04b2a-9d67-45d7-a4f9-d366f15eb139 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:38.947112 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:38.948081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.948293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.948563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.949278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:38.949664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:38.951255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:38.967340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 973/1955] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:38.968671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:38.978050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bff53-a087-4323-8904-d9ca344fbaf3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:38.982714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:38.983352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:38.983615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:38.983893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:38.984562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:38.986526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.013706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 983/1956] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:38 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:39.017818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.029427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a6d3-b71c-4f56-bba5-63ff46e6332f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.088268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.088893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.089176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.089455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.090169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.092096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.112351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 974/1957] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:39.116417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.127343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34141-5070-4630-bb1b-8a0120e1ee6f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.132516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.133157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.133439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.133811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.134832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.136481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.149945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 984/1958] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:39.151586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.164101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130dfd1-7a00-42e3-a819-1d01748b6ff9 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.169062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.169362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.169753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.170054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.170748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.172677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8d1ca-c326-4885-b7ac-caa46dd8b00d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.185466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 975/1959] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:39.189478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.205021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e4d05-f0ba-4314-8473-b4e9148e9fa9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.211589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.212307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.212585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.212862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] PATH_INFO: `/v3/projects/601eee48942943a79828cdb28d0f8bf4/users/be629951ec574e6f81f1eec0db1a59fe/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.231024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=601eee48942943a79828cdb28d0f8bf4, user_id=be629951ec574e6f81f1eec0db1a59fe, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.234024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.254334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 985/1960] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:39 2022] PUT /identity/v3/projects/601eee48942943a79828cdb28d0f8bf4/users/be629951ec574e6f81f1eec0db1a59fe/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:39.266141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.266441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.266721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.298404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] MFA Rules not processed for user `be629951ec574e6f81f1eec0db1a59fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:39.299662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:39.375881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 976/1961] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:39.380337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.476466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bccab-3651-4155-a280-e59fe22ec592 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.481308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.481993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.482283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.482811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.483507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.485515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.521913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 986/1962] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:39.524289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.539498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86de3b-4119-4243-99ce-933eb099abce None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.571872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:39.639195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.640177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.640552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.640929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.654075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.654075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.675958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 977/1963] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:39.679824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.687196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:39.698967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d151-cd7b-4e1c-942a-6fd6a11bb1f9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.704497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.705198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.705721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.706188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.706977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.709179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.725302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 987/1964] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:39.729167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.740234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af904ad-ba3f-4460-8076-04256707a65c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.746968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.748006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.748391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.748774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.749528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.751373 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c4ed2-f2fe-4f5d-a98f-61d0da6ea20a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.762769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 978/1965] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:39.767221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:39.778484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3480-355e-44a2-940a-4497aa1c8005 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:39.784800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:39.785564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.786030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.786314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] PATH_INFO: `/v3/projects/d2658601e7c44db29c31ef6b6d33b74a/users/ff2d7b28834740f1b86b17af8fa19c23/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.802340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2658601e7c44db29c31ef6b6d33b74a, user_id=ff2d7b28834740f1b86b17af8fa19c23, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:39.805121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:39.820149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 988/1966] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:39 2022] PUT /identity/v3/projects/d2658601e7c44db29c31ef6b6d33b74a/users/ff2d7b28834740f1b86b17af8fa19c23/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:39.831103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:39.831776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:39.831776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:39.865110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] MFA Rules not processed for user `ff2d7b28834740f1b86b17af8fa19c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:39.866549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:39.960533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 979/1967] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:39.964907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.000012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.001620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:40.064262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c9d00-fb15-4d71-99e0-033d3ab4d714 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.076991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.077648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.077963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.078375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.078918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.080895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320d690-30ab-4afc-87ee-15b1b8417e84 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.081085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.087143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.087794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.088080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.088359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.090258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:40.122777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 989/1968] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:39 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:40.125374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.171412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:40.190087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.191881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.207435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 980/1969] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:40.250260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c372-04f7-4384-a723-953947a5881a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.256206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.256856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.257139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.257414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.258125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.260094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.294564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 990/1970] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:40 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:40.299249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.313770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e81a9-e8f5-4f20-819b-c7f82a4b0133 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.329135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.329870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.330162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.330446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.331127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.335764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.357833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 981/1971] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:40.362124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.379821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572035-d42d-4146-b0c0-521e2755c339 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.388377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.389306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.390064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.390485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] PATH_INFO: `/v3/projects/8c2dea567f2a4e9881136c42201d0443/users/c5ffbcf75c3a4129b1e0fe32249972f7/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.416702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2dea567f2a4e9881136c42201d0443, user_id=c5ffbcf75c3a4129b1e0fe32249972f7, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.420779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.451007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 991/1972] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:40 2022] PUT /identity/v3/projects/8c2dea567f2a4e9881136c42201d0443/users/c5ffbcf75c3a4129b1e0fe32249972f7/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:40.457017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.468969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ac2f-3ff2-4a4f-9774-5794fdeed23c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.605007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.606106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.607296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.611288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.612155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.614669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167d87-434a-4e60-bffc-c1e1a8ce09cb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.625744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 982/1973] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:40.630867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.648044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b93b-59a7-43f7-baa7-c07f3376da37 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.654076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.655031 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.655531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.657740 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] PATH_INFO: `/v3/projects/8c2dea567f2a4e9881136c42201d0443/users/c5ffbcf75c3a4129b1e0fe32249972f7/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.660622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2dea567f2a4e9881136c42201d0443, user_id=c5ffbcf75c3a4129b1e0fe32249972f7, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:40.665717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:40.694700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 992/1974] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:40 2022] PUT /identity/v3/projects/8c2dea567f2a4e9881136c42201d0443/users/c5ffbcf75c3a4129b1e0fe32249972f7/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:40.707576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08541-5fbd-4845-b347-03442f7ede45 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.709063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08541-5fbd-4845-b347-03442f7ede45 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.709351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08541-5fbd-4845-b347-03442f7ede45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:40.750031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6ea08541-5fbd-4845-b347-03442f7ede45 None None] MFA Rules not processed for user `c5ffbcf75c3a4129b1e0fe32249972f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:40.758194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6ea08541-5fbd-4845-b347-03442f7ede45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:40.882345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 983/1975] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:40.895946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:40.900944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:40.965088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f18f9-9bdf-43ef-8caa-b549dcd480c2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:40.969487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:40.970676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:40.970676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:40.970676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:41.039948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:41.041973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af478da-82af-4f19-9573-79819906d261 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:41.063697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 993/1976] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:41.597266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:41.597818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:41.598272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:41.599849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 984/1977] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:41.651167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3225c33-2d97-4952-b927-0afc1d242621 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:41.651730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3225c33-2d97-4952-b927-0afc1d242621 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:41.651730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3225c33-2d97-4952-b927-0afc1d242621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:41.672488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e3225c33-2d97-4952-b927-0afc1d242621 None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:41.673862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e3225c33-2d97-4952-b927-0afc1d242621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:41.762949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 994/1978] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:12:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:41.781589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:41.783395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:41.815362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad853-1bab-4d8b-829b-4c94346ba463 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:41.824771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:41.826253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:41.826713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:41.827140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:41.840271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:41.842430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b92f4-cd87-4dac-8448-42821d8a9f7c service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:41.862862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 985/1979] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:42.637125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:42.637413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:42.637692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:42.639750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 995/1980] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:12:42.687942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:42.688459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:42.688725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:42.745901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] MFA Rules not processed for user `4ad9d639d97b415aba4f3772e8ea0f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:42.750670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:42.785793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:42.787620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:42.789541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 986/1981] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:12:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:42.816163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:42.836181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60603939-9e5d-4d94-9e22-7cc2717d2ea3 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:42.850579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:42.851367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:42.856124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:42.856565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:42.884456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:42.886724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:42.922471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 987/1982] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:42.939546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:42.944270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:42.963181 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13a327-f86e-48cf-b4de-c6e969de80ff service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:42.976461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:42.977879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:42.978307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:42.978685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:43.011844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:43.019352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d007d-6574-41d9-b7d1-e1b4320df2f7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:43.024295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:43.030012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:43.031051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:43.033020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:43.033630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] PATH_INFO: `/v3/users/7fb19714f22c490cbacfa339cb44570b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:43.041242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb19714f22c490cbacfa339cb44570b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:43.043580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:43.118186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:43.120376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:43.133196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 988/1983] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:43.156564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb19714f22c490cbacfa339cb44570b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.173259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb19714f22c490cbacfa339cb44570b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.183905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb19714f22c490cbacfa339cb44570b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.184911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb19714f22c490cbacfa339cb44570b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.185696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb19714f22c490cbacfa339cb44570b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.203032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 996/1984] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:42 2022] DELETE /identity/v3/users/7fb19714f22c490cbacfa339cb44570b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:43.607198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:43.627106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e06e-6575-4ffe-a96f-cb31d0a43d51 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:43.759307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:43.760111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:43.760286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:43.760869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] PATH_INFO: `/v3/projects/bb12343d6f014f9895125e19d02b4849` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:43.763728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb12343d6f014f9895125e19d02b4849)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:43.767053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:43.847599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb12343d6f014f9895125e19d02b4849', 'request_id': 'req-d480df93-d6d7-47a6-bb5c-4d8c392f732c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.856836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb12343d6f014f9895125e19d02b4849', 'request_id': 'req-d480df93-d6d7-47a6-bb5c-4d8c392f732c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.867191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb12343d6f014f9895125e19d02b4849', 'request_id': 'req-d480df93-d6d7-47a6-bb5c-4d8c392f732c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.884063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] The token cache is being invalidate because project bb12343d6f014f9895125e19d02b4849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:43.884488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.885027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:43.889489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 989/1985] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:43 2022] DELETE /identity/v3/projects/bb12343d6f014f9895125e19d02b4849 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:43.890807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:43.892532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:43.909132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6fdd-a427-4695-b502-a589fa0c6511 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:43.985823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:43.986623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:43.987001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:43.987390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] PATH_INFO: `/v3/users/44f4385235b945528aff09549c237d3b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:43.988686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f4385235b945528aff09549c237d3b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:43.991757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.013239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.014775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:44.044685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f4385235b945528aff09549c237d3b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.058977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f4385235b945528aff09549c237d3b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.066810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f4385235b945528aff09549c237d3b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.068179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f4385235b945528aff09549c237d3b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.068831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f4385235b945528aff09549c237d3b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.071440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:44.088965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480df93-d6d7-47a6-bb5c-4d8c392f732c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.089728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 997/1986] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:43 2022] DELETE /identity/v3/users/44f4385235b945528aff09549c237d3b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:44.096101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:44.150764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.151396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.151677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.151951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.153723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:44.219536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.221003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69283ec-6380-4ffd-bd3e-604f668dd2e4 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.231027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 990/1987] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:44.270155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.272355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:44.356959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408a2c3-6329-43f8-a1b2-a5946968be9f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.361690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.362395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.362660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.363052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] PATH_INFO: `/v3/projects/e2ee154316de4a72b654f83030a9a1fc` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.364323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ee154316de4a72b654f83030a9a1fc)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.366329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.413479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ee154316de4a72b654f83030a9a1fc', 'request_id': 'req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.420456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ee154316de4a72b654f83030a9a1fc', 'request_id': 'req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.426457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ee154316de4a72b654f83030a9a1fc', 'request_id': 'req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.436775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] The token cache is being invalidate because project e2ee154316de4a72b654f83030a9a1fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:44.437088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.437718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a83f3369-cc98-4ac7-81a3-0ca48b9fed5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:44.439258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 998/1988] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:44 2022] DELETE /identity/v3/projects/e2ee154316de4a72b654f83030a9a1fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:44.445548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.445864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.446153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.466668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:44.467882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:44.527554 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 991/1989] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:44.536596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.536883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.537161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.561724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:44.563003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:44.572300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 999/1990] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:44.577311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.588056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737bcf1-29cc-49e1-b910-16b26db1d449 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.593596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.594398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.594812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.595140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.595996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:44.596344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.598113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.624937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 992/1991] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:44.626090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.637383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2e369-bd68-4d64-a11b-942e817c5f79 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.643655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.644384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.644734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.645140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.645928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.648051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.677830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1000/1992] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:44 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:44.682604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.693484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4cd3-3f9e-47f3-8a72-c6466949262e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.734267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:44.750154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.750798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.751076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.751348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.751994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.753949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.777885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 993/1993] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:44 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:44.780044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.790358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db4ea-68c1-44c4-a854-d59ca450680d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.794978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.795597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.795877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.796150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.796849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.798529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.808836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1001/1994] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:44.812718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.823999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031affb3-050f-4855-b5df-f037a9094791 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.829071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.829817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.830134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.830415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.831119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.832717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9259f7b-7b3e-42b0-9bef-eeff412bb9b2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.846264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 994/1995] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:44.850480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:44.860558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757680a-2778-4883-9cc9-a1d37a278d11 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:44.866375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:44.867226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.868110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.868724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] PATH_INFO: `/v3/projects/6323b5a9d0184acf9e31d9e4890c0800/users/b803a6988ceb4d62a09db9d1f9914f4b/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.888000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6323b5a9d0184acf9e31d9e4890c0800, user_id=b803a6988ceb4d62a09db9d1f9914f4b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:44.890828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:44.914842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1002/1996] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:44 2022] PUT /identity/v3/projects/6323b5a9d0184acf9e31d9e4890c0800/users/b803a6988ceb4d62a09db9d1f9914f4b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:44.916781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:44.917070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:44.917387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:44.948991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] MFA Rules not processed for user `b803a6988ceb4d62a09db9d1f9914f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:44.950520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:45.025254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 995/1997] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:45.037479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:45.040102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:45.068438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:45.112520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f214a-fa25-4ddb-ad84-a4842b8c30cd admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:45.117530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:45.118303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:45.118590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:45.118871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:45.128162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:45.129960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8750a7-1ff1-40e2-944e-8db7c2f320cf service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:45.140586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1003/1998] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:45.693559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:45.695148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:45.765209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58cf0-c465-4ed5-ae1a-9c27ef9bdc18 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:45.769833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:45.770581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:45.770861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:45.771133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] PATH_INFO: `/v3/users/0f67a8d9a7614e73bdfa0eca8497aea3` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:45.772282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f67a8d9a7614e73bdfa0eca8497aea3)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:45.774230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:45.820302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f67a8d9a7614e73bdfa0eca8497aea3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:45.820956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f67a8d9a7614e73bdfa0eca8497aea3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:45.826333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:45.834752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f67a8d9a7614e73bdfa0eca8497aea3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:45.842078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f67a8d9a7614e73bdfa0eca8497aea3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:45.848968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f67a8d9a7614e73bdfa0eca8497aea3'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:45.854952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 996/1999] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:45 2022] DELETE /identity/v3/users/0f67a8d9a7614e73bdfa0eca8497aea3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:45.859331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:45.859651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:45.859955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:45.878934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:45.880656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:45.940920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1004/2000] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:45.952207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:45.954138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:46.014403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bbace-c682-4f65-ac58-1dfca9880642 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:46.019069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:46.019698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:46.020007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:46.020827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:46.031733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:46.034096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:46.044271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 997/2001] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:46.336432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:46.338559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:46.354529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f8ad-8c12-45a4-b7e5-71e3d7d8532f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:46.359129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:46.359787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:46.360071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:46.360351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] PATH_INFO: `/v3/projects/0ebf2c8c9aaf4750aeee33df1cb559a4` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:46.361523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebf2c8c9aaf4750aeee33df1cb559a4)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:46.363495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:46.416311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebf2c8c9aaf4750aeee33df1cb559a4', 'request_id': 'req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.424292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebf2c8c9aaf4750aeee33df1cb559a4', 'request_id': 'req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.431589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebf2c8c9aaf4750aeee33df1cb559a4', 'request_id': 'req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.442969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] The token cache is being invalidate because project 0ebf2c8c9aaf4750aeee33df1cb559a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:46.443308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.443963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.445490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1005/2002] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:46 2022] DELETE /identity/v3/projects/0ebf2c8c9aaf4750aeee33df1cb559a4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:46.449617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:46.460271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854cf3b-6e88-444b-9f10-fc5e61f07b31 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:46.519594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:46.520309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:46.520561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:46.520836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] PATH_INFO: `/v3/users/ab3ee8d36f78460c8415bb2c8a82e816` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:46.522055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3ee8d36f78460c8415bb2c8a82e816)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:46.524017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:46.611848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3ee8d36f78460c8415bb2c8a82e816'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.612734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3ee8d36f78460c8415bb2c8a82e816'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.630894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3ee8d36f78460c8415bb2c8a82e816'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.647560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3ee8d36f78460c8415bb2c8a82e816'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.655379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:46.664266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8d89605b-ee82-4706-b1f8-58b4e1dbd889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3ee8d36f78460c8415bb2c8a82e816'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:46.672779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 998/2003] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:46 2022] DELETE /identity/v3/users/ab3ee8d36f78460c8415bb2c8a82e816 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:47.130315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.132411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:47.156194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2ce9f-40e6-4e27-b5b7-a96caa794f14 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.226645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:47.243432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.244196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.244555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.244836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] PATH_INFO: `/v3/projects/1ba87303a0364e22b32c9f442697649a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.246314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba87303a0364e22b32c9f442697649a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.248413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.302170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba87303a0364e22b32c9f442697649a', 'request_id': 'req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:47.311262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba87303a0364e22b32c9f442697649a', 'request_id': 'req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:47.318169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba87303a0364e22b32c9f442697649a', 'request_id': 'req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:47.331534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] The token cache is being invalidate because project 1ba87303a0364e22b32c9f442697649a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:47.331827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:47.332514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9fb1fdbc-13e4-4a84-959d-5c0e540ad060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:47.334496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1006/2004] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:47 2022] DELETE /identity/v3/projects/1ba87303a0364e22b32c9f442697649a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:47.341544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.342139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.342232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.363228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:47.365020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:47.429089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 999/2005] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:47.438513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.438725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.439001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.459854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:47.461052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:47.469424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1007/2006] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:47.473905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.484141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659d39-996b-4410-aeff-bc5d6556354f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.488814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.489439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.489716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.490020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.490719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:47.491021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.492900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.505116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1000/2007] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:47.509615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.519041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bbfa8-da09-4788-95b6-55b8a844db74 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.524270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.524884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.525161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.525435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.526151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.528177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.556665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1008/2008] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:47 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:47.562028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.573603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc3eb6-52f9-4d50-b64d-8f66ef45ad82 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.641629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.642587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.642905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.643145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.643834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.645793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.672837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1001/2009] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:47 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:47.678125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.693543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0f21-3325-4087-8d98-4834c97f1a07 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.701018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.701727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.702047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.702380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.703254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.704938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.717565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1009/2010] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:47.722084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.733362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14905ce6-0420-4b46-9d6c-a60a1ed44951 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.738892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.739710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.739997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.740273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.742236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.749441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416ffc6-94fd-454d-af83-b5b16ee27104 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.768362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:47.775390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1002/2011] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:47.783711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bcac-e9c0-413b-8e33-48002d173a16 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:47.793415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:47.794152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.794441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.794736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] PATH_INFO: `/v3/projects/0c2d090ae01d4931aed9607acad3cf41/users/94fce447109c44e7b3298253d3d3c88d/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.813856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2d090ae01d4931aed9607acad3cf41, user_id=94fce447109c44e7b3298253d3d3c88d, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:47.817419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:47.847831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1010/2012] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:47 2022] PUT /identity/v3/projects/0c2d090ae01d4931aed9607acad3cf41/users/94fce447109c44e7b3298253d3d3c88d/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:47.859439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:47.859734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:47.860011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:47.907346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] MFA Rules not processed for user `94fce447109c44e7b3298253d3d3c88d`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:47.909445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:47.991973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:48.026809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1003/2013] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:48.070494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:48.072284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.168465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fbad5-00d6-481d-bfc4-bb2af6acb200 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:48.168465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:48.168465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.168465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.169259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.187502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:48.189240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59440c00-3442-4480-963c-bb08e8eaa20f service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:48.204836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1011/2014] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:48.222842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:48.294094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ac91b-5423-45d5-a266-776d8905c9fc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:48.299073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:48.299809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.300192 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.300566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] PATH_INFO: `/v3/users/94fce447109c44e7b3298253d3d3c88d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.302561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fce447109c44e7b3298253d3d3c88d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:48.304668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:48.357469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fce447109c44e7b3298253d3d3c88d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.358200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fce447109c44e7b3298253d3d3c88d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.370246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fce447109c44e7b3298253d3d3c88d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.378035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fce447109c44e7b3298253d3d3c88d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.388232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fce447109c44e7b3298253d3d3c88d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.394465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1004/2015] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:48 2022] DELETE /identity/v3/users/94fce447109c44e7b3298253d3d3c88d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:48.401093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.401573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.401914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.421479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:48.422254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:48.423237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.423430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.432924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:48.491906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1012/2016] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:48.497474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604957c-1129-4567-8dd7-29e521c2e99d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:48.502267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:48.502552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:48.503216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.503499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.503774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] PATH_INFO: `/v3/users/b803a6988ceb4d62a09db9d1f9914f4b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.504047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.505165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b803a6988ceb4d62a09db9d1f9914f4b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:48.507131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:48.518993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:48.577876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55905cc7-b7a8-4b66-b0a8-ed2a5dc3b7f7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:48.578092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b803a6988ceb4d62a09db9d1f9914f4b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.578984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b803a6988ceb4d62a09db9d1f9914f4b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.582411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:48.583112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.583401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.583677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.592929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b803a6988ceb4d62a09db9d1f9914f4b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.594026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:48.595602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c2aea-679f-4958-bd29-fc2727d5d112 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:48.614854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1013/2017] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:48.616618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b803a6988ceb4d62a09db9d1f9914f4b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.627343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b803a6988ceb4d62a09db9d1f9914f4b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:48.638101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1005/2018] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:48 2022] DELETE /identity/v3/users/b803a6988ceb4d62a09db9d1f9914f4b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:48.645223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299323-3802-4029-8493-0045e69ead4f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.645620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299323-3802-4029-8493-0045e69ead4f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.645929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95299323-3802-4029-8493-0045e69ead4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.669196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-95299323-3802-4029-8493-0045e69ead4f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:48.670983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-95299323-3802-4029-8493-0045e69ead4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.753125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1014/2019] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:48.762542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:48.764666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:48.841052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df46-b62c-4584-827b-a796d08761c8 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:48.845651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:48.846746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:48.846746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:48.847103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:48.864214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:48.865725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647e670-45ef-4505-b214-43ef2b79fa24 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:48.876785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1006/2020] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:49.075446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299323-3802-4029-8493-0045e69ead4f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.076885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-95299323-3802-4029-8493-0045e69ead4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.089824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95299323-3802-4029-8493-0045e69ead4f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.094661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.095323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.095604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.095877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] PATH_INFO: `/v3/projects/0c2d090ae01d4931aed9607acad3cf41` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.097061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2d090ae01d4931aed9607acad3cf41)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.099038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.215524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2d090ae01d4931aed9607acad3cf41', 'request_id': 'req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.220243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:49.225640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2d090ae01d4931aed9607acad3cf41', 'request_id': 'req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.240184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2d090ae01d4931aed9607acad3cf41', 'request_id': 'req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.251700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] The token cache is being invalidate because project 0c2d090ae01d4931aed9607acad3cf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:49.252065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.252934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f0eeb6bd-cca5-456b-ba1f-24af6b39f776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.256343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1015/2021] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:49 2022] DELETE /identity/v3/projects/0c2d090ae01d4931aed9607acad3cf41 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:49.260835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.261196 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.261567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.282365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:49.283578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.381944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1007/2022] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:49.389667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.390175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.390653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.400100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.403055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.415201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:49.416401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.422304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfeb563-bb2d-4d4a-8a76-7471ecec7eab None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.432769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.435526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.436199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.438164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1016/2023] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:49.438405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] PATH_INFO: `/v3/projects/6323b5a9d0184acf9e31d9e4890c0800` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.439459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.440477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6323b5a9d0184acf9e31d9e4890c0800)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.441053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.443155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.454796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093eef3-b2e2-4658-9b87-02f68aac0129 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.459900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.460544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.460824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.461097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] PATH_INFO: `/v3/users/be629951ec574e6f81f1eec0db1a59fe` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.462308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be629951ec574e6f81f1eec0db1a59fe)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.464266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.478310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:49.500662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6323b5a9d0184acf9e31d9e4890c0800', 'request_id': 'req-8eb6e095-2257-423c-b6c8-50914f11046a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.507685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6323b5a9d0184acf9e31d9e4890c0800', 'request_id': 'req-8eb6e095-2257-423c-b6c8-50914f11046a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.514131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6323b5a9d0184acf9e31d9e4890c0800', 'request_id': 'req-8eb6e095-2257-423c-b6c8-50914f11046a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.514710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be629951ec574e6f81f1eec0db1a59fe'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.524768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] The token cache is being invalidate because project 6323b5a9d0184acf9e31d9e4890c0800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:49.525063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.526054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.526609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be629951ec574e6f81f1eec0db1a59fe'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.527589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1008/2024] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:49 2022] DELETE /identity/v3/projects/6323b5a9d0184acf9e31d9e4890c0800 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:12:49.528450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.529780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.534703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be629951ec574e6f81f1eec0db1a59fe'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.535554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be629951ec574e6f81f1eec0db1a59fe'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.537343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3100bb09-7ced-41fd-a204-eac2fd51a044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be629951ec574e6f81f1eec0db1a59fe'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:49.554518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1017/2025] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:49 2022] DELETE /identity/v3/users/be629951ec574e6f81f1eec0db1a59fe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:49.554518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fae3-bc7d-4999-ade2-87abb986f0bf None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.554518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fae3-bc7d-4999-ade2-87abb986f0bf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.554518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fae3-bc7d-4999-ade2-87abb986f0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.572648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8bb9fae3-bc7d-4999-ade2-87abb986f0bf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:49.574276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8bb9fae3-bc7d-4999-ade2-87abb986f0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.595365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e095-2257-423c-b6c8-50914f11046a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.634914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1018/2026] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:49.636211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee989-f217-4ccb-94a9-f415f580938c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.636501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee989-f217-4ccb-94a9-f415f580938c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.636847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee989-f217-4ccb-94a9-f415f580938c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.656876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2d6ee989-f217-4ccb-94a9-f415f580938c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:49.658130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2d6ee989-f217-4ccb-94a9-f415f580938c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.658618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.659245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.659590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.659923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.661121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:49.661427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.663100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.667033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1019/2027] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:49.668276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.668560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.668832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.675834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1009/2028] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:49.677486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.679161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.690522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:49.691772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.700597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1020/2029] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:49.701496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.702882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:49.714935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093eea4c-1cad-42e1-a1e4-1f26be143dd9 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.719621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.720236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.720519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.720800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.721463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.723940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.735743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251823d-2b56-4c8f-8df2-84ca3e94d0e6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.739974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.740593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.740870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.741145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.750810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.751816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1021/2030] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:49.752323 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.752491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.758052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:49.763265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1010/2031] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:49.763992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.768715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:49.777207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305db9-d5ce-4889-a770-7fb78959bf3b service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.825594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd776-ee89-4255-bae1-653b5ee2f37b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.830885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.831525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.831862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.832139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.833401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:12:49.833713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.835410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.850351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1022/2032] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:12:49.854614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.854754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.855990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.856348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.856693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.857446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.859957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.866562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bfdcb-2109-4363-98c4-b50901710bd8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.872189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.873090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.873680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.873905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.874956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.878332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.902019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1011/2033] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:49.903307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.914606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfb289-2f39-4d43-aacd-955518ceefa0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:49.928549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:49.929574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:49.930167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:49.930612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:49.933515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:49.935571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:49.942576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1023/2034] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:12:49.948688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.952500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1012/2035] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:49.953422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:49.966504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a79c0-91e7-46f9-a9b8-e3559f517b25 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.042030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.043577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.044014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.044440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.045489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.047879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.051925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1808-3659-4592-b424-849a96d3002f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.057283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.058806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.059324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.059768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.060589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.062797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.079275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1013/2036] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:50.080619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.101213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca46a7-0258-4e2c-af75-659de622634b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.115404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.116164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.116420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1024/2037] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:12:49 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:12:50.116592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.117279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] PATH_INFO: `/v3/projects/707034727c6c44eb8cea2f5951ac48b7/users/99c077c212f1472d85df56c05a51ceb9/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.120383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.144256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707034727c6c44eb8cea2f5951ac48b7, user_id=99c077c212f1472d85df56c05a51ceb9, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.148745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe37b33-e3bb-450e-b707-84147e32534c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.150228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621388d-7218-4e92-b8c9-e4c17d033d39 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.157346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.158324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.158583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.158858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.159576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.161756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.173390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1014/2038] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:50 2022] PUT /identity/v3/projects/707034727c6c44eb8cea2f5951ac48b7/users/99c077c212f1472d85df56c05a51ceb9/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:50.178374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.178776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.179183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.183896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1025/2039] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:12:50.185056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.197673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78195c-4628-4fb0-9d80-eea9ce5f4e27 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.200722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:50.202085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.269099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:50.295902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.299960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1015/2040] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:50.300165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.300165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.300322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.300417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.300417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.300567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.300650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.306924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:50.312209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cdccd-5368-4922-972c-ab678d550d67 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.316205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1026/2041] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:12:50.317711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.319643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.320267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.320387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.320576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.320854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] PATH_INFO: `/v3/projects/601eee48942943a79828cdb28d0f8bf4` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.322091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=601eee48942943a79828cdb28d0f8bf4)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.324359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.374176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601eee48942943a79828cdb28d0f8bf4', 'request_id': 'req-f5d7a740-10b5-4486-b542-7dd7f96f7b36'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.385926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fbe8f-6c31-4a5a-8e67-5c10c4415059 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.390017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601eee48942943a79828cdb28d0f8bf4', 'request_id': 'req-f5d7a740-10b5-4486-b542-7dd7f96f7b36'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.399177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601eee48942943a79828cdb28d0f8bf4', 'request_id': 'req-f5d7a740-10b5-4486-b542-7dd7f96f7b36'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.411420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] The token cache is being invalidate because project 601eee48942943a79828cdb28d0f8bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:50.412024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.412864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.414861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1016/2042] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:50 2022] DELETE /identity/v3/projects/601eee48942943a79828cdb28d0f8bf4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:50.415737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.417187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.439660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.440288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.440578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.440855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.442776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.492914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7a740-10b5-4486-b542-7dd7f96f7b36 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.497903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.498690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.499037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.499321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] PATH_INFO: `/v3/projects/39a7fbb2f3ca405bbaa394825b4a3e73/users/79e4add99b984c25b04553927bed7cb5/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.515628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a7fbb2f3ca405bbaa394825b4a3e73, user_id=79e4add99b984c25b04553927bed7cb5, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.518306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.523604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.525097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302b7aa-16a3-43cc-9f91-9010d71a7917 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.536177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1017/2043] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:12:50 2022] PUT /identity/v3/projects/39a7fbb2f3ca405bbaa394825b4a3e73/users/79e4add99b984c25b04553927bed7cb5/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:50.537022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.538685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.540264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1027/2044] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:50.541611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.541918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.542198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.564507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:50.566041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.614672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:50.623730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f74f-4356-4ee3-8fa6-cc88f9cccbed admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.630935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.631785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.632155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.632531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] PATH_INFO: `/v3/users/ff2d7b28834740f1b86b17af8fa19c23` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.634361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2d7b28834740f1b86b17af8fa19c23)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.636627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.637473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1028/2045] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:50.647669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:50.649642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:50.660100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:50.695991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2d7b28834740f1b86b17af8fa19c23'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.696705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2d7b28834740f1b86b17af8fa19c23'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.713500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2d7b28834740f1b86b17af8fa19c23'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.722131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727a24b-ef18-42d9-ad78-30f58915e98c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:50.728118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:50.729296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:50.729607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:50.730383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:50.733123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2d7b28834740f1b86b17af8fa19c23'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.740951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:50.742094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2d7b28834740f1b86b17af8fa19c23'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:50.742650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:50.748767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1018/2046] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:50 2022] DELETE /identity/v3/users/ff2d7b28834740f1b86b17af8fa19c23 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:50.753617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1029/2047] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:12:51.298412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:51.299891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:51.391736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b946-a5e9-48bf-9da2-331b2e2aec78 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:51.397126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:51.397868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:51.398153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:51.398427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] PATH_INFO: `/v3/projects/d2658601e7c44db29c31ef6b6d33b74a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:51.399578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2658601e7c44db29c31ef6b6d33b74a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:51.401502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:51.451481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2658601e7c44db29c31ef6b6d33b74a', 'request_id': 'req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.459771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2658601e7c44db29c31ef6b6d33b74a', 'request_id': 'req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.466522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2658601e7c44db29c31ef6b6d33b74a', 'request_id': 'req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.477675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] The token cache is being invalidate because project d2658601e7c44db29c31ef6b6d33b74a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:51.478518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.479239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7c407cfc-47f2-4347-a1f9-68b93cdc6ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.481060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1019/2048] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:51 2022] DELETE /identity/v3/projects/d2658601e7c44db29c31ef6b6d33b74a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:51.485294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:51.495422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04678c1-b278-4b27-9bd7-8b50a3bdf7c7 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:51.553714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:51.554390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:51.554725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:51.554940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] PATH_INFO: `/v3/users/c5ffbcf75c3a4129b1e0fe32249972f7` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:51.556138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ffbcf75c3a4129b1e0fe32249972f7)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:51.558129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:51.608331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ffbcf75c3a4129b1e0fe32249972f7'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.620348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ffbcf75c3a4129b1e0fe32249972f7'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.627432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ffbcf75c3a4129b1e0fe32249972f7'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.628137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ffbcf75c3a4129b1e0fe32249972f7'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.628815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ffbcf75c3a4129b1e0fe32249972f7'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:51.643447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1030/2049] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:12:51 2022] DELETE /identity/v3/users/c5ffbcf75c3a4129b1e0fe32249972f7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:51.839051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:51.839396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:51.839609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:51.849286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:51.890837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] MFA Rules not processed for user `99c077c212f1472d85df56c05a51ceb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:51.892058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:51.991111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1020/2050] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:51.999021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:52.002562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:52.085709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:52.125364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9227fc-95a6-4276-ba96-79561e6ba6c4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:52.130574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:52.131343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:52.131853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:52.132408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:52.144712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:52.146424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c576553-d2f0-40a7-8a48-e4dc34b14032 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:52.157513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1031/2051] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:52.251464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:52.252943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:52.290066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:52.290355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:52.290691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:52.347749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] MFA Rules not processed for user `79e4add99b984c25b04553927bed7cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:52.350277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:52.389430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668987df-99d1-4a73-9875-ccae2dcef9fb None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:52.396594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:52.397360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:52.397550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:52.397831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] PATH_INFO: `/v3/projects/8c2dea567f2a4e9881136c42201d0443` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:52.400245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2dea567f2a4e9881136c42201d0443)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:52.402713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:52.477620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1032/2052] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:52.490347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:12:52.492087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:52.500358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2dea567f2a4e9881136c42201d0443', 'request_id': 'req-eb23fc57-93c3-4339-964f-a8e89ea2afa4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:52.511061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2dea567f2a4e9881136c42201d0443', 'request_id': 'req-eb23fc57-93c3-4339-964f-a8e89ea2afa4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:52.513636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32562f7-9cb9-4e2e-b304-033f928bab53 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:12:52.520419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2dea567f2a4e9881136c42201d0443', 'request_id': 'req-eb23fc57-93c3-4339-964f-a8e89ea2afa4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:52.540630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] The token cache is being invalidate because project 8c2dea567f2a4e9881136c42201d0443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:12:52.541298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:52.542716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb23fc57-93c3-4339-964f-a8e89ea2afa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:12:52.545304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1021/2053] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:12:52 2022] DELETE /identity/v3/projects/8c2dea567f2a4e9881136c42201d0443 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:12:52.658383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:12:52.659012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:52.659298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:52.661592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:52.664647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:52.807987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:12:52.809609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043df5a-bafb-4c7f-b131-b780c2ad8690 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:12:52.821864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1033/2054] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:12:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:12:54.838950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:54.839515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:54.839515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:54.863197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] MFA Rules not processed for user `99c077c212f1472d85df56c05a51ceb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:54.864699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:54.888982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:12:54.945782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1022/2055] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:12:55.305127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd028-1bab-4abf-a021-07247555f063 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:12:55.305610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd028-1bab-4abf-a021-07247555f063 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:12:55.305699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fd028-1bab-4abf-a021-07247555f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:12:55.330061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-949fd028-1bab-4abf-a021-07247555f063 None None] MFA Rules not processed for user `79e4add99b984c25b04553927bed7cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:12:55.331395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-949fd028-1bab-4abf-a021-07247555f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:12:55.339122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1034/2056] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:02.542994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:02.544732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:02.651033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54aec7-be87-4cd8-a50f-784c89d0c63f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:02.655813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:02.656469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:02.656753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:02.657031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] PATH_INFO: `/v3/users/816bcf956bb442e39ff9feb7e606dab4` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:02.658254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=816bcf956bb442e39ff9feb7e606dab4)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:02.660251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:02.713646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816bcf956bb442e39ff9feb7e606dab4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:02.716874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816bcf956bb442e39ff9feb7e606dab4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:02.730369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816bcf956bb442e39ff9feb7e606dab4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:02.738563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816bcf956bb442e39ff9feb7e606dab4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:02.745998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-1e4a28cd-dc5a-426a-8d6a-7b07267db6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816bcf956bb442e39ff9feb7e606dab4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:02.752013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1023/2057] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:02 2022] DELETE /identity/v3/users/816bcf956bb442e39ff9feb7e606dab4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:02.943683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd028-1bab-4abf-a021-07247555f063 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:02.964247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fd028-1bab-4abf-a021-07247555f063 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:02.975248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:03.027571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.028309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.028584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.028854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] PATH_INFO: `/v3/projects/ba0c305435a042aabbb61816788d7c66` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.030075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0c305435a042aabbb61816788d7c66)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.031979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.080794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0c305435a042aabbb61816788d7c66', 'request_id': 'req-e1044062-b142-468b-8766-e723e20d4318'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:03.089294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0c305435a042aabbb61816788d7c66', 'request_id': 'req-e1044062-b142-468b-8766-e723e20d4318'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:03.095856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0c305435a042aabbb61816788d7c66', 'request_id': 'req-e1044062-b142-468b-8766-e723e20d4318'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:03.106065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] The token cache is being invalidate because project ba0c305435a042aabbb61816788d7c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:03.106352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:03.106979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e1044062-b142-468b-8766-e723e20d4318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:03.108488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1035/2058] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:02 2022] DELETE /identity/v3/projects/ba0c305435a042aabbb61816788d7c66 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:03.113407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.113798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.114114 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.133336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:03.134566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:03.181787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:03.199494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1024/2059] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:03.208960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.209242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.209516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.229981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:03.231208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:03.240554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1036/2060] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:03.244832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.255302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd9ac9-e360-4579-8542-850dc1fe8d98 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.260712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.261329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.261605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.261903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.262606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:03.262906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.264506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.277336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1025/2061] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:13:03.282809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.292208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710e36-11f8-4f65-ba95-34c7b2252a9e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.296831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.297443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.297720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.298025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.298699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.301802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.330106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1037/2062] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:03.334551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.344763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd34c70-aa8f-4684-b29b-37d609fc7ed2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.402443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.403068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.403351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.403623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.404327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.406268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.429245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1026/2063] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:03.432964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.443913 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91365-3b2e-487f-9ae1-c9a7d282e7b4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.448794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.449412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.449692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.449999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.450697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.452295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.462654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1038/2064] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:03.466718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.477439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a09a-40ba-4901-ae74-1e78d8f404b7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.482773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.483447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.483730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.484031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.484759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.486410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa37ba-563d-40d9-ae52-ae57657494ed admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.497934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1027/2065] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:03.502571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.514661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602d966-8b76-4cb9-b283-ef7ba02d292f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.519289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.519946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.520225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.520500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] PATH_INFO: `/v3/projects/f70eb0c5a5e34533ad5ff90a5cf76aa3/users/50d598819d6845f58d6d93dd6ccc8437/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.536317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70eb0c5a5e34533ad5ff90a5cf76aa3, user_id=50d598819d6845f58d6d93dd6ccc8437, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.539102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.552707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1039/2066] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:03 2022] PUT /identity/v3/projects/f70eb0c5a5e34533ad5ff90a5cf76aa3/users/50d598819d6845f58d6d93dd6ccc8437/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:03.563810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.564385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.564682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.594365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] MFA Rules not processed for user `50d598819d6845f58d6d93dd6ccc8437`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:03.595500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:03.662492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1028/2067] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:03.667263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.709225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:03.736702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890cff0-8a52-4fee-b093-23b66c72ac8c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.742348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.743009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.743289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.743563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.744742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.747603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.777535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1040/2068] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:03.781962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.793105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd1fde-e7b4-41b1-b6fe-93dcc2b6d0b7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.849195 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:03.868175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.869269 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.869723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.870027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.870779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.872717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.896117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1029/2069] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:03 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:03.900100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.911988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeee2a5-f673-4c12-a6c6-9ad299a7cc15 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.917369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.918152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.918502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.918909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.919755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.921446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.937049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1041/2070] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:13:03.941359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:03.952598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b030-f7f9-4cc2-845a-4e7d6820ff64 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:03.958471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:03.959490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:03.959856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:03.960241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] PATH_INFO: `/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1/users/09f7fe9c9cdb4657ba5cad37705d3b44/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:03.976527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc1f12bf4f24429aeb7a2ee7a3db0c1, user_id=09f7fe9c9cdb4657ba5cad37705d3b44, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:03.979391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:03.993359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1030/2071] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:03 2022] PUT /identity/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1/users/09f7fe9c9cdb4657ba5cad37705d3b44/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:03.997364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:04.010269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce25a0-c984-4bf1-8fe1-decb2ce33032 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:04.073773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:04.074520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:04.074902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:04.075495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:04.076105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:04.078428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f688b-5310-48c0-972f-89bf08fdcd7b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:04.097080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1042/2072] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:04.101976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:04.113339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f92f6-c02c-4b60-911a-373ef56802b3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:04.119132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:04.119845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:04.120123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:04.121721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] PATH_INFO: `/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1/users/09f7fe9c9cdb4657ba5cad37705d3b44/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:04.122638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc1f12bf4f24429aeb7a2ee7a3db0c1, user_id=09f7fe9c9cdb4657ba5cad37705d3b44, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:04.125281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:04.137168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1031/2073] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:04 2022] PUT /identity/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1/users/09f7fe9c9cdb4657ba5cad37705d3b44/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:04.149387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:04.149781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:04.150170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:04.184863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] MFA Rules not processed for user `09f7fe9c9cdb4657ba5cad37705d3b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:04.186121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:04.260982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1043/2074] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:04.273307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:04.275202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:04.335045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3a20-3aaa-4115-9d09-d7cdc3f16a3e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:04.338907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:04.339533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:04.339814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:04.340089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:04.350782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:04.352279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:04.362313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1032/2075] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:04.436943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:04.447109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233bb93-aed8-4da9-8c35-baecf33f6e6b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:04.450958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:04.451568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:04.451845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:04.452116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:04.484022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:04.512447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:04.513990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630574-71f9-4fd2-89fe-d6eb954614bc service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:04.523435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1044/2076] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:07.236277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:07.296327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8baf-8cfb-4d89-b420-d6b992b5e4ee service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:07.300949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:07.301589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:07.301894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:07.302170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] PATH_INFO: `/v3/users/50d598819d6845f58d6d93dd6ccc8437` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:07.303349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d598819d6845f58d6d93dd6ccc8437)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:07.305287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:07.347489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:07.353727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d598819d6845f58d6d93dd6ccc8437'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:07.354674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d598819d6845f58d6d93dd6ccc8437'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:07.367815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d598819d6845f58d6d93dd6ccc8437'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:07.375047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d598819d6845f58d6d93dd6ccc8437'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:07.382173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d598819d6845f58d6d93dd6ccc8437'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:07.388966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1033/2077] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:07 2022] DELETE /identity/v3/users/50d598819d6845f58d6d93dd6ccc8437 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:13:07.393484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:07.394248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:07.394248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:07.414671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:07.415881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:07.482883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1045/2078] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:07.492610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:07.494152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:07.559874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149992c-215c-4299-bdb5-784e4a41908b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:07.563990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:07.564751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:07.565035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:07.565313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:07.577637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:07.579180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:07.591067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1034/2079] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:07.987034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:07.988528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:08.001772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e40b0-01f3-469d-9091-fabe9dd15d08 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.006518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.007169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.007452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.007733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] PATH_INFO: `/v3/projects/f70eb0c5a5e34533ad5ff90a5cf76aa3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.008893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70eb0c5a5e34533ad5ff90a5cf76aa3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.010898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:08.056863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70eb0c5a5e34533ad5ff90a5cf76aa3', 'request_id': 'req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.063513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70eb0c5a5e34533ad5ff90a5cf76aa3', 'request_id': 'req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.069572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70eb0c5a5e34533ad5ff90a5cf76aa3', 'request_id': 'req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.080009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] The token cache is being invalidate because project f70eb0c5a5e34533ad5ff90a5cf76aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:08.080300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.080958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.083028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1046/2080] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:07 2022] DELETE /identity/v3/projects/f70eb0c5a5e34533ad5ff90a5cf76aa3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:08.087346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:08.096747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097699-3920-47e6-9cf7-c8ea2b01c8be service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.151387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.151951 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.152235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.152512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] PATH_INFO: `/v3/users/09f7fe9c9cdb4657ba5cad37705d3b44` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.153670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f7fe9c9cdb4657ba5cad37705d3b44)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.155640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:08.201529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f7fe9c9cdb4657ba5cad37705d3b44'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.202212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f7fe9c9cdb4657ba5cad37705d3b44'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.214421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:08.215634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f7fe9c9cdb4657ba5cad37705d3b44'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.223754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f7fe9c9cdb4657ba5cad37705d3b44'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.230792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-7eaba2d6-5d62-4e83-b9da-ceef0d94f111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f7fe9c9cdb4657ba5cad37705d3b44'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.236665 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1035/2081] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:08 2022] DELETE /identity/v3/users/09f7fe9c9cdb4657ba5cad37705d3b44 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:08.510920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:08.512396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:08.526069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe795ef-cae1-4aa3-b759-9cd88ec1d1b1 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.575506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:08.586256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.587153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.587682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.587966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] PATH_INFO: `/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.589171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc1f12bf4f24429aeb7a2ee7a3db0c1)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.591131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:08.636392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc1f12bf4f24429aeb7a2ee7a3db0c1', 'request_id': 'req-cfea7b34-2856-43b2-817b-1e6ed17ab248'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.645647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc1f12bf4f24429aeb7a2ee7a3db0c1', 'request_id': 'req-cfea7b34-2856-43b2-817b-1e6ed17ab248'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.651862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc1f12bf4f24429aeb7a2ee7a3db0c1', 'request_id': 'req-cfea7b34-2856-43b2-817b-1e6ed17ab248'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.662620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] The token cache is being invalidate because project 1fc1f12bf4f24429aeb7a2ee7a3db0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:08.662907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.663537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-cfea7b34-2856-43b2-817b-1e6ed17ab248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:08.665082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1047/2082] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:08 2022] DELETE /identity/v3/projects/1fc1f12bf4f24429aeb7a2ee7a3db0c1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:08.670111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aa55b-108b-41a9-b650-05506578728a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.670399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aa55b-108b-41a9-b650-05506578728a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.670677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aa55b-108b-41a9-b650-05506578728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.689570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c72aa55b-108b-41a9-b650-05506578728a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:08.690727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c72aa55b-108b-41a9-b650-05506578728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:08.748592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1036/2083] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:08.756809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17b95-f233-4953-9473-030c746820be None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.757301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17b95-f233-4953-9473-030c746820be None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.757583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17b95-f233-4953-9473-030c746820be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.779833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5ea17b95-f233-4953-9473-030c746820be None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:08.781057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5ea17b95-f233-4953-9473-030c746820be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:08.789985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1048/2084] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:08.794204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aa55b-108b-41a9-b650-05506578728a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:08.804492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aa55b-108b-41a9-b650-05506578728a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.810239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.810916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.811144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.811424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.812173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:08.812609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.814348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:08.829577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1037/2085] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:13:08.835253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17b95-f233-4953-9473-030c746820be None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:08.845814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17b95-f233-4953-9473-030c746820be None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.852611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.853249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.853530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.853806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.854963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.856906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:08.886822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1049/2086] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:08 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:08.890621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:08.902731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ecc5-bfa5-4a70-9d8f-28497b152932 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:08.986687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:08.987801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:08.988268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:08.988781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:08.989776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:08.992345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.021451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1038/2087] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:08 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:09.026009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.037400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb3cfc-d0f0-4954-8e8d-630da57f536d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.042486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.043230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.043576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.043924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.044714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.046541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.059594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1050/2088] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:09.064017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.078345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d7ab-d77c-498c-932a-65b8a68fe490 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.084000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.084718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.085067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.085411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.086225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.087961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a797fbe-0f70-4b4c-aee4-88ba74bc50e6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.099781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1039/2089] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:13:09.103855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.115070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3249606-2f6d-4aa3-a0df-7d3cc95dc477 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.119807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.120489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.120772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.121049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] PATH_INFO: `/v3/projects/711b0d7dc317496b846dc149628bdbf1/users/9a8cef578b624533ab3c1dd1c6b1d138/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.136589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=711b0d7dc317496b846dc149628bdbf1, user_id=9a8cef578b624533ab3c1dd1c6b1d138, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.139418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.153294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1051/2090] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:09 2022] PUT /identity/v3/projects/711b0d7dc317496b846dc149628bdbf1/users/9a8cef578b624533ab3c1dd1c6b1d138/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:09.165171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.165455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.165730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.196214 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] MFA Rules not processed for user `9a8cef578b624533ab3c1dd1c6b1d138`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:09.197462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:09.233240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:09.267854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1040/2091] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:09.272984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.346719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b8636-4226-4e7a-9752-0abe7c174d5a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.352102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.352738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.353022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.353299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.354030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.355956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.399009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1052/2092] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:09 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:09.403526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.436826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8f891-eedd-449e-b88c-71cdda9dd264 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.502740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.503461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.503809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.505085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.505502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.507566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.531382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1041/2093] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:09 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:13:09.535342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.543606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:09.549218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993060e6-85fd-4187-8eff-16bb313124e3 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.557372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.558258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.558734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.559343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.560361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.563426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.581312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1053/2094] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:13:09.582779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.593611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890b87-0938-4bf8-a24a-1ad5c6815636 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.598428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.599229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.599513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.599787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] PATH_INFO: `/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9/users/eca980a39a1f43469ac56b4fe07641e6/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.616479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ed6a0e93f94d0a989a6cf2482545f9, user_id=eca980a39a1f43469ac56b4fe07641e6, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.619276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.641207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1042/2095] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:09 2022] PUT /identity/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9/users/eca980a39a1f43469ac56b4fe07641e6/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:09.648278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.664204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321c7a1-d3bc-4b9c-bc9e-6c1be3e680cf admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.746317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.747214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.747720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.748221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.750580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.753943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8497269-340d-424d-b93e-23d414baa4e0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.773882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1054/2096] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:09.777378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.788728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff73c8-9e4b-42c9-aaac-02b24f5e3b6d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.793382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.794079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.794365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.794676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] PATH_INFO: `/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9/users/eca980a39a1f43469ac56b4fe07641e6/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.796623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ed6a0e93f94d0a989a6cf2482545f9, user_id=eca980a39a1f43469ac56b4fe07641e6, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:09.799290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:09.810624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1043/2097] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:09 2022] PUT /identity/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9/users/eca980a39a1f43469ac56b4fe07641e6/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:13:09.821204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.821564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.821963 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:09.850083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] MFA Rules not processed for user `eca980a39a1f43469ac56b4fe07641e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:09.851284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:09.917941 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1055/2098] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:09.929885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:09.931634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:09.988972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650523a3-830d-47d9-9e7f-26bd81e6ac2e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:09.992708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:09.993330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:09.993613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:09.993925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:10.034743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:10.049669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:10.051688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4037ea-d1e8-418a-aa25-a8af04e4abb7 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:10.060540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1044/2099] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:16.745261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:16.756798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311825-4a0e-4fd3-9f4a-cd967b45efc8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:16.760757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:16.761382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:16.761663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:16.761970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:16.771126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:16.772562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:16.782366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1056/2100] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:20.056584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=8 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:26.778011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=72 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:31.422628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a408e-f9e7-4e83-91f6-f3967ee5cd3b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:31.423313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a408e-f9e7-4e83-91f6-f3967ee5cd3b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:31.423313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a408e-f9e7-4e83-91f6-f3967ee5cd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:31.445402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-100a408e-f9e7-4e83-91f6-f3967ee5cd3b None None] MFA Rules not processed for user `99c077c212f1472d85df56c05a51ceb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:31.446730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-100a408e-f9e7-4e83-91f6-f3967ee5cd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:31.508371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1045/2101] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:41.498574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=25 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:54.128407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:54.191596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b1388-7819-438c-a086-67b307b0f4d5 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:54.196197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:54.196852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:54.197134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:54.197412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] PATH_INFO: `/v3/users/9a8cef578b624533ab3c1dd1c6b1d138` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:54.198649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8cef578b624533ab3c1dd1c6b1d138)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:54.200847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:54.247046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8cef578b624533ab3c1dd1c6b1d138'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.258584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8cef578b624533ab3c1dd1c6b1d138'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.265605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8cef578b624533ab3c1dd1c6b1d138'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.266315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8cef578b624533ab3c1dd1c6b1d138'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.266934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8cef578b624533ab3c1dd1c6b1d138'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.280239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1057/2102] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:54 2022] DELETE /identity/v3/users/9a8cef578b624533ab3c1dd1c6b1d138 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:54.284676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:54.285146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:54.285246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:54.304442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:54.305640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:54.363806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1046/2103] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:54.373499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:54.374920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:54.430826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2304e1-c952-40a0-b41a-caa7d1fef868 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:54.434391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:54.435006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:54.435285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:54.435558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:54.444503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:54.445961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:54.455698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1058/2104] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:54.712142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:54.713656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:54.725523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a3699-9070-428a-9687-a8acefccd568 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:54.734641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:54.736340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:54.736966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:54.737246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] PATH_INFO: `/v3/projects/711b0d7dc317496b846dc149628bdbf1` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:54.738480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=711b0d7dc317496b846dc149628bdbf1)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:54.740409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:54.787688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711b0d7dc317496b846dc149628bdbf1', 'request_id': 'req-6e7be6a8-b2fc-434b-9950-f98c2252c55c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.795263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711b0d7dc317496b846dc149628bdbf1', 'request_id': 'req-6e7be6a8-b2fc-434b-9950-f98c2252c55c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.801711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711b0d7dc317496b846dc149628bdbf1', 'request_id': 'req-6e7be6a8-b2fc-434b-9950-f98c2252c55c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.812020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] The token cache is being invalidate because project 711b0d7dc317496b846dc149628bdbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:54.812309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.812965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.814645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1047/2105] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:54 2022] DELETE /identity/v3/projects/711b0d7dc317496b846dc149628bdbf1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:54.818981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:54.829064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681967c-107e-4cad-928d-df00e71aadcb service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:54.852008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:54.906572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:54.907699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:54.907983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:54.908373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] PATH_INFO: `/v3/users/eca980a39a1f43469ac56b4fe07641e6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:54.917922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca980a39a1f43469ac56b4fe07641e6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:54.917922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:54.979567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca980a39a1f43469ac56b4fe07641e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:54.995487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca980a39a1f43469ac56b4fe07641e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.003426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca980a39a1f43469ac56b4fe07641e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.004178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca980a39a1f43469ac56b4fe07641e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.004888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9c1d7eb2-1f34-4e65-bc1d-f8a8d9ddc13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca980a39a1f43469ac56b4fe07641e6'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.022423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1059/2106] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:54 2022] DELETE /identity/v3/users/eca980a39a1f43469ac56b4fe07641e6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:55.327179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.328786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:55.344588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7be6a8-b2fc-434b-9950-f98c2252c55c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.411680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.412405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.412633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.412908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] PATH_INFO: `/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.414151 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ed6a0e93f94d0a989a6cf2482545f9)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.416046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.445257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:55.463465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ed6a0e93f94d0a989a6cf2482545f9', 'request_id': 'req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.470294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ed6a0e93f94d0a989a6cf2482545f9', 'request_id': 'req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.476430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ed6a0e93f94d0a989a6cf2482545f9', 'request_id': 'req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.486762 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] The token cache is being invalidate because project 84ed6a0e93f94d0a989a6cf2482545f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:55.487050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.487674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b3a04461-bf60-40bd-a98d-ce8832cb2cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:55.489171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1048/2107] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:55 2022] DELETE /identity/v3/projects/84ed6a0e93f94d0a989a6cf2482545f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:55.494180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.494648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.494739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.514535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:55.515730 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:55.576527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1060/2108] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:55.585971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.586258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.586535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.605999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:55.607228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:55.615149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1049/2109] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:55.619397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.628886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84bb5-2279-4a2e-a5ae-1c7075fa58d4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.634060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.634669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.634948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.635247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.635949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:55.636253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.638090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.650060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1061/2110] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:13:55.654249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.663275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc866-078c-4696-ae45-8419036d5486 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.667748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.668342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.668618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.668889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.669536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.671435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.697967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1050/2111] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:55 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:55.702199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.713473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee881e-eb0d-422b-8734-671439475c4c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.742001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:55.773411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.774056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.774339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.774615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.775282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.777232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.796476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1062/2112] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:55 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:55.800575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.811226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806c003-dea4-4b12-8121-3e71a12d878f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.815835 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.816442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.816722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.816998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.817690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.819938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.829812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1051/2113] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:55.833398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.843069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942880bb-5971-4f85-bc90-8dbd0b4cad6f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.847581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.848181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.848461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.848734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.849420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.851030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2e7d-6ad2-47a3-9b4d-1284026698ca admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.862077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1063/2114] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:55.867511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:55.878042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa9ad-4740-4015-9b4f-2613c5029efd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:55.883279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:55.883939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.884220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.884493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] PATH_INFO: `/v3/projects/235ee1f2f1844ad9b40198fe4e46346d/users/e9a931f625444496a145a154517a1cf8/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.899049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=235ee1f2f1844ad9b40198fe4e46346d, user_id=e9a931f625444496a145a154517a1cf8, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:55.901678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:55.914273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1052/2115] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:55 2022] PUT /identity/v3/projects/235ee1f2f1844ad9b40198fe4e46346d/users/e9a931f625444496a145a154517a1cf8/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:55.925246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:55.925632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:55.926184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:55.955386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] MFA Rules not processed for user `e9a931f625444496a145a154517a1cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:55.956604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.030232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1064/2116] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:13:56.034758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.098574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f28a-e36b-4482-82c1-95212ee5c59c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.103802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.104423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.104700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.104971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.105729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.107668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.136273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1053/2117] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:56 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:56.140272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.151816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fdd58-d3f2-46ec-a812-5d528f05aff1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.213617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.214184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.214460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.214732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.215404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.217330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.240232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1065/2118] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:56 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:56.241435 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.251741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ed12-ab52-4f2e-be6f-3da76b944382 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.256926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.257528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.257808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.258111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.258799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.260393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.270749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1054/2119] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:56.275416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.285156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794cc40-a648-4ecd-9f86-4efe6039229d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.289658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.290352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.290632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.290907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] PATH_INFO: `/v3/projects/87cf7500027548ea8034dbfff1b11b28/users/5e91191c2d8a42218c5b40529ca42ef5/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.305071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=87cf7500027548ea8034dbfff1b11b28, user_id=5e91191c2d8a42218c5b40529ca42ef5, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.307982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.320802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1066/2120] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:56 2022] PUT /identity/v3/projects/87cf7500027548ea8034dbfff1b11b28/users/5e91191c2d8a42218c5b40529ca42ef5/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:56.325093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.335784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9db6-0980-4dcd-bd40-e21aa92079d3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.355315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.395699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.396940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.397219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.397489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.398213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.399867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6e7c3-78ea-48ee-90da-cda6a1e0e295 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.409409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1055/2121] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:56.413716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.419522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.425542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41324f-d3e5-4d1a-9037-3c33c750c6de admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.431838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.432847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.433226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.434178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] PATH_INFO: `/v3/projects/87cf7500027548ea8034dbfff1b11b28/users/5e91191c2d8a42218c5b40529ca42ef5/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.438042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87cf7500027548ea8034dbfff1b11b28, user_id=5e91191c2d8a42218c5b40529ca42ef5, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.442880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.455635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1067/2122] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:56 2022] PUT /identity/v3/projects/87cf7500027548ea8034dbfff1b11b28/users/5e91191c2d8a42218c5b40529ca42ef5/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:13:56.466795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.467025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.467300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.497870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] MFA Rules not processed for user `5e91191c2d8a42218c5b40529ca42ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:56.499096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.589395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1056/2123] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:56.600889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.602923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.666601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06a746-765b-4151-be49-e2bb070154f2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.670487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.671118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.671400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.671677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.681039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.682534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e11a0-e23f-4404-a133-531cca43860c service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.691962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1068/2124] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:56.760005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:56.821336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb07e0b-9599-4f1d-94d0-9218f6431fda None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:56.826255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:56.826899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.827179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.827491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] PATH_INFO: `/v3/users/e9a931f625444496a145a154517a1cf8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.828655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a931f625444496a145a154517a1cf8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:56.830607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:56.877430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a931f625444496a145a154517a1cf8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:56.878105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a931f625444496a145a154517a1cf8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:56.879230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:56.891754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a931f625444496a145a154517a1cf8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:56.901035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a931f625444496a145a154517a1cf8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:56.907719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a931f625444496a145a154517a1cf8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:56.913507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1057/2125] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:56 2022] DELETE /identity/v3/users/e9a931f625444496a145a154517a1cf8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:56.918033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253690-89ed-4469-8297-8f8854023e82 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:56.918325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253690-89ed-4469-8297-8f8854023e82 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:56.918611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253690-89ed-4469-8297-8f8854023e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:56.937312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-18253690-89ed-4469-8297-8f8854023e82 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:56.938535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-18253690-89ed-4469-8297-8f8854023e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:56.997615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1069/2126] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:57.008109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:57.009517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:57.065937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257a9a9-b7d6-4e0e-b4ad-2c041f9c7e41 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:57.070088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:57.070697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:57.070975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:57.071245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:57.080543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:57.081996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:57.091967 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1058/2127] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:57.376451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253690-89ed-4469-8297-8f8854023e82 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:57.377938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-18253690-89ed-4469-8297-8f8854023e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:57.393480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253690-89ed-4469-8297-8f8854023e82 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:57.399127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:57.399778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:57.399991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:57.400270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] PATH_INFO: `/v3/projects/235ee1f2f1844ad9b40198fe4e46346d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:57.401448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] RBAC: Authorizing `identity:delete_project(project_id=235ee1f2f1844ad9b40198fe4e46346d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:57.403487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:57.449861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235ee1f2f1844ad9b40198fe4e46346d', 'request_id': 'req-ad324872-cc56-4b67-8c02-291ec5cb602d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.456323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235ee1f2f1844ad9b40198fe4e46346d', 'request_id': 'req-ad324872-cc56-4b67-8c02-291ec5cb602d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.462505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235ee1f2f1844ad9b40198fe4e46346d', 'request_id': 'req-ad324872-cc56-4b67-8c02-291ec5cb602d'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.472461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] The token cache is being invalidate because project 235ee1f2f1844ad9b40198fe4e46346d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:57.472741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.473367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.474891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1070/2128] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:57 2022] DELETE /identity/v3/projects/235ee1f2f1844ad9b40198fe4e46346d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:57.478809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:57.488386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bbcdd-fbb7-4596-b0d6-7b076624611a service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:57.546714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:57.547348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:57.547623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:57.547894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] PATH_INFO: `/v3/users/5e91191c2d8a42218c5b40529ca42ef5` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:57.549056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e91191c2d8a42218c5b40529ca42ef5)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:57.551029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:57.596360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e91191c2d8a42218c5b40529ca42ef5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.597014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e91191c2d8a42218c5b40529ca42ef5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.608480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e91191c2d8a42218c5b40529ca42ef5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.612027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:57.615660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e91191c2d8a42218c5b40529ca42ef5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.624502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3b459cb1-f359-4707-b78e-5f1208c859f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e91191c2d8a42218c5b40529ca42ef5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:57.631709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1059/2129] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:13:57 2022] DELETE /identity/v3/users/5e91191c2d8a42218c5b40529ca42ef5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:57.907199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:57.908690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:57.926242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324872-cc56-4b67-8c02-291ec5cb602d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:57.940347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:57.986355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:57.987479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:57.987763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:57.988040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] PATH_INFO: `/v3/projects/87cf7500027548ea8034dbfff1b11b28` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:57.989233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87cf7500027548ea8034dbfff1b11b28)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:57.991744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.036278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87cf7500027548ea8034dbfff1b11b28', 'request_id': 'req-15681717-3c0a-4355-adef-a58c7020dae9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:58.045747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87cf7500027548ea8034dbfff1b11b28', 'request_id': 'req-15681717-3c0a-4355-adef-a58c7020dae9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:58.052636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87cf7500027548ea8034dbfff1b11b28', 'request_id': 'req-15681717-3c0a-4355-adef-a58c7020dae9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:58.064117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] The token cache is being invalidate because project 87cf7500027548ea8034dbfff1b11b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:58.064415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:58.065052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-15681717-3c0a-4355-adef-a58c7020dae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:58.066692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1071/2130] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:57 2022] DELETE /identity/v3/projects/87cf7500027548ea8034dbfff1b11b28 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:58.071489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.071781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.072057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.092220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:58.093341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:58.151777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1060/2131] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:58.160416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.160705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.160979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.181740 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:58.182982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:58.190919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1072/2132] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:58.195203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.205311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535458-95b5-40dc-9996-de66c1a174a1 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.210352 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.210949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.211224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.211492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.212178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:58.212475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.214085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.226056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1061/2133] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:13:58.230311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.239549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e32b38-7ff6-4a74-a25e-b35aae230f80 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.244166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.244774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.245057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.245341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.246050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.247961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.274893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1073/2134] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:58.278958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.289730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21775-7d5d-44bb-a2ce-db5751917497 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.344239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.344833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.345120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.345421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.346148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.348075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.370092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1062/2135] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:58.374225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.385379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aeec8d-ac6c-4bd5-b9a5-9ea7b13ed055 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.390123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.390745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.391027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.391307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.392009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.393633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.407874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1074/2136] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:58.409347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.419334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be4ffe-9a7e-4948-b5bf-3ca0e7a8179f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.424552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.425167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.425444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.425714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.426440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.428082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bbec6-ba66-4010-bd7a-b5c3a70ce083 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.439549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1063/2137] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:58.444359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.455123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dad6c-0a8b-4b4e-b25c-9a93985a4d22 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.461022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.461696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.462010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.462289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] PATH_INFO: `/v3/projects/b2627016c5454d74a6a4f57755fbfc4e/users/d9ee3c2ba6f24622a614f483ae737bc0/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.478054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2627016c5454d74a6a4f57755fbfc4e, user_id=d9ee3c2ba6f24622a614f483ae737bc0, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.480827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.494102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1075/2138] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:58 2022] PUT /identity/v3/projects/b2627016c5454d74a6a4f57755fbfc4e/users/d9ee3c2ba6f24622a614f483ae737bc0/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:58.504617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.505135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.505316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.535640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] MFA Rules not processed for user `d9ee3c2ba6f24622a614f483ae737bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:58.536888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:58.577512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:58.600970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1064/2139] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:58.605532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.650601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:58.669041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce10e1-3e81-4d89-bdbc-1fb0b90718a2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.673713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.674364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.674649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.674927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.675632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.677553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.703578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1076/2140] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:58.707750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.718790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0530d1-e193-4063-a244-24d383edfa75 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.773627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.774271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.774558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.774832 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.775501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.777412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.797833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1065/2141] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:13:58 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:58.801874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.812790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472af07-8435-4467-abe7-0880583abeee admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.817449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.818054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.818901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.819291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.819999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.821593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.832620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1077/2142] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:58.837071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.847016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403f852-4b54-4230-9e7f-f014f5da76d0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.851766 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.852350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.852634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.852915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] PATH_INFO: `/v3/projects/467dced9a80340ffb7ec5ce6f13fa349/users/f0616918fefa41219a122a9e925b0a4c/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.869389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] RBAC: Authorizing `identity:create_grant(project_id=467dced9a80340ffb7ec5ce6f13fa349, user_id=f0616918fefa41219a122a9e925b0a4c, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.872078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.887137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1066/2143] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:58 2022] PUT /identity/v3/projects/467dced9a80340ffb7ec5ce6f13fa349/users/f0616918fefa41219a122a9e925b0a4c/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:58.891194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.901003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c043e-5895-41ea-9f27-e058a79c3f94 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.957981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.959014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.959014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.959218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.959849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:58.961442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d530d4c-5035-4b18-9cc3-f65835949667 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:58.972038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1078/2144] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:13:58.979452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:58.989601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d22461-64cd-4018-99e1-33be021c430f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:58.994814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:58.995473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:58.995755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:58.996033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] PATH_INFO: `/v3/projects/467dced9a80340ffb7ec5ce6f13fa349/users/f0616918fefa41219a122a9e925b0a4c/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:58.997997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467dced9a80340ffb7ec5ce6f13fa349, user_id=f0616918fefa41219a122a9e925b0a4c, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.000603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.011318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1067/2145] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:13:58 2022] PUT /identity/v3/projects/467dced9a80340ffb7ec5ce6f13fa349/users/f0616918fefa41219a122a9e925b0a4c/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:13:59.022176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.022466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.022876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.050658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] MFA Rules not processed for user `f0616918fefa41219a122a9e925b0a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:13:59.051886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:59.122208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1079/2146] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:13:59.135016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.136701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:13:59.192100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f4b54-8446-4d89-bec3-4d85efcc53e0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.195814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.196421 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.196702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.196975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.206060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.207487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.216731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1068/2147] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:13:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:13:59.526442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.535646 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c94c63-398a-49af-aaa4-f007c18d7c20 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.545180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0616918fefa41219a122a9e925b0a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '467dced9a80340ffb7ec5ce6f13fa349', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.545825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.546141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.546418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.547244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:59.547425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.549028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.562070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1080/2148] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:13:59.566365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.576812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c862f7-9744-49a4-9323-cf099fc10b8a service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.581432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0616918fefa41219a122a9e925b0a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '467dced9a80340ffb7ec5ce6f13fa349', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.582072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.582353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.582648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.583311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.586175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.615970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1069/2149] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:13:59 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:13:59.620052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.629925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78659106-14e1-49bc-adcf-6f74831c0d89 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.652969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:59.692081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0616918fefa41219a122a9e925b0a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '467dced9a80340ffb7ec5ce6f13fa349', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.692707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.692989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.693298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.694033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:13:59.694352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.695938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.708067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1081/2150] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:13:59.746295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:13:59.761235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f595b-d513-46f2-8855-c1e36ecf3c8f tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:13:59.764959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0616918fefa41219a122a9e925b0a4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '467dced9a80340ffb7ec5ce6f13fa349', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:13:59.765605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:13:59.765995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:13:59.766270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] PATH_INFO: `/v3/projects/8b14005e9490439ca3e8a9a1784432d5` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:13:59.767419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8b14005e9490439ca3e8a9a1784432d5)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:13:59.769348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:13:59.781761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:13:59.818282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b14005e9490439ca3e8a9a1784432d5', 'request_id': 'req-86a9ea77-94be-4938-a527-fe46815cafb2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:59.826120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b14005e9490439ca3e8a9a1784432d5', 'request_id': 'req-86a9ea77-94be-4938-a527-fe46815cafb2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:59.833105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b14005e9490439ca3e8a9a1784432d5', 'request_id': 'req-86a9ea77-94be-4938-a527-fe46815cafb2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:59.844616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] The token cache is being invalidate because project 8b14005e9490439ca3e8a9a1784432d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:13:59.844973 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:59.845713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-86a9ea77-94be-4938-a527-fe46815cafb2 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:13:59.847526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1070/2151] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:13:59 2022] DELETE /identity/v3/projects/8b14005e9490439ca3e8a9a1784432d5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:00.058231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:00.061872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:00.133172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc81b-b326-4742-b8b0-2db4b8a5ff98 tempest-VolumeQuotaClassesTest-1198685249 tempest-VolumeQuotaClassesTest-1198685249-project-admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:00.197396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:00.198626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:00.198626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:00.198885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] PATH_INFO: `/v3/users/d9ee3c2ba6f24622a614f483ae737bc0` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:00.200619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ee3c2ba6f24622a614f483ae737bc0)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:00.202812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:00.255712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ee3c2ba6f24622a614f483ae737bc0'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.268459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ee3c2ba6f24622a614f483ae737bc0'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.270522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:00.277548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ee3c2ba6f24622a614f483ae737bc0'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.278575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ee3c2ba6f24622a614f483ae737bc0'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.279382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ee3c2ba6f24622a614f483ae737bc0'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.292664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1082/2152] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:14:00 2022] DELETE /identity/v3/users/d9ee3c2ba6f24622a614f483ae737bc0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:00.297176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e39ac-2737-4999-9630-110394117073 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:00.297490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e39ac-2737-4999-9630-110394117073 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:00.297764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e39ac-2737-4999-9630-110394117073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:00.316434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a37e39ac-2737-4999-9630-110394117073 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:00.317870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a37e39ac-2737-4999-9630-110394117073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:00.376209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1071/2153] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:00.385870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:00.387360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:00.443716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960d1f-8de9-418c-b614-92573e6a030f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:00.447415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:00.448037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:00.448319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:00.448593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:00.457827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:00.459780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:00.469110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1083/2154] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:00.737689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e39ac-2737-4999-9630-110394117073 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:00.739222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a37e39ac-2737-4999-9630-110394117073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:00.753396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e39ac-2737-4999-9630-110394117073 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:00.759060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:00.760781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:00.761153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:00.761328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] PATH_INFO: `/v3/projects/b2627016c5454d74a6a4f57755fbfc4e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:00.762632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2627016c5454d74a6a4f57755fbfc4e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:00.764530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:00.812293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2627016c5454d74a6a4f57755fbfc4e', 'request_id': 'req-5d4b94a6-67cf-49b5-b608-6fa1380283d6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.820360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2627016c5454d74a6a4f57755fbfc4e', 'request_id': 'req-5d4b94a6-67cf-49b5-b608-6fa1380283d6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.826988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2627016c5454d74a6a4f57755fbfc4e', 'request_id': 'req-5d4b94a6-67cf-49b5-b608-6fa1380283d6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.837681 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] The token cache is being invalidate because project b2627016c5454d74a6a4f57755fbfc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:14:00.837996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.838645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.840155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1072/2155] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:14:00 2022] DELETE /identity/v3/projects/b2627016c5454d74a6a4f57755fbfc4e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:00.844238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:00.853718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa368a7-481c-42d5-ae7f-ed223291710f service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:00.924632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:00.925311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:00.925595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:00.925901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] PATH_INFO: `/v3/users/f0616918fefa41219a122a9e925b0a4c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:00.927136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0616918fefa41219a122a9e925b0a4c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:00.929090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:00.978571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0616918fefa41219a122a9e925b0a4c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.990719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0616918fefa41219a122a9e925b0a4c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.997707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0616918fefa41219a122a9e925b0a4c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.998432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0616918fefa41219a122a9e925b0a4c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:00.999038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-57d2a20f-d21c-422b-beee-39cde49cf6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0616918fefa41219a122a9e925b0a4c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.012795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1084/2156] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:14:00 2022] DELETE /identity/v3/users/f0616918fefa41219a122a9e925b0a4c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:01.335479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.337023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:01.363584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b94a6-67cf-49b5-b608-6fa1380283d6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.385143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:01.426081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.426740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.427081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.427357 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] PATH_INFO: `/v3/projects/467dced9a80340ffb7ec5ce6f13fa349` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.429150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=467dced9a80340ffb7ec5ce6f13fa349)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.431076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.485443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467dced9a80340ffb7ec5ce6f13fa349', 'request_id': 'req-739f3859-016c-417f-bedd-770776120d04'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.494622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467dced9a80340ffb7ec5ce6f13fa349', 'request_id': 'req-739f3859-016c-417f-bedd-770776120d04'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.501179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467dced9a80340ffb7ec5ce6f13fa349', 'request_id': 'req-739f3859-016c-417f-bedd-770776120d04'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.511710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] The token cache is being invalidate because project 467dced9a80340ffb7ec5ce6f13fa349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:14:01.511988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.512650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-739f3859-016c-417f-bedd-770776120d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:01.514261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1073/2157] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:14:01 2022] DELETE /identity/v3/projects/467dced9a80340ffb7ec5ce6f13fa349 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:01.520301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.520905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.521654 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.535906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:01.543319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:01.545309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:01.609614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1085/2158] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:01.618921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.619213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.619494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.639221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:01.640425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:01.648477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1074/2159] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:01.652687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.663187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d388f-a2ce-483e-bc56-acf210a20242 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.667950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.668871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.669149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.669416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.670463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:14:01.670766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.672427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.689656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1086/2160] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:14:01.690672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.701118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a0fb3-3066-4d9d-a9a8-3e33b1cbd49a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.705985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.706552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.706833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.707111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.707784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.710025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.737970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1075/2161] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:14:01 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:01.743366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.753911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcf1c8-d9cd-4779-9c72-5004d220b547 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.816191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.816821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.817116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.817393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.818163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.820743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.846010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1087/2162] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:14:01 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:01.847263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.859530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276d01d-5355-41b9-96a7-bcc4e1715ae3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.864290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.864929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.865216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.865495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.866372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.867980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.879526 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1076/2163] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:01.884072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.894528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38b97f-28d0-4085-a9ba-b6ec6fc95cf8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.900061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.900653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.900927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.901196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.901950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.903635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a01e-da22-4ae7-b572-0d86b4bd64e5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.925295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1088/2164] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:01.925597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:01.932583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc65fc-41d1-462b-9080-26327c639421 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:01.942669 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:01.943699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.944119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.944401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] PATH_INFO: `/v3/projects/05e1761d372a41a78c201d95ef0909f5/users/61d55322f9e348baa681004bc5ee9ccb/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:01.962374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e1761d372a41a78c201d95ef0909f5, user_id=61d55322f9e348baa681004bc5ee9ccb, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:01.965172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:01.978510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1077/2165] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:01 2022] PUT /identity/v3/projects/05e1761d372a41a78c201d95ef0909f5/users/61d55322f9e348baa681004bc5ee9ccb/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:01.988785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c876-849e-4de8-89cb-c6c363bfabdc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:01.989080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c876-849e-4de8-89cb-c6c363bfabdc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:01.989362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c876-849e-4de8-89cb-c6c363bfabdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:02.016796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d4f1c876-849e-4de8-89cb-c6c363bfabdc None None] MFA Rules not processed for user `61d55322f9e348baa681004bc5ee9ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:02.017973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d4f1c876-849e-4de8-89cb-c6c363bfabdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:02.085449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1089/2166] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:02.096297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:02.098060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:02.154521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:02.187788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5073dd-f3d1-4213-84d8-020e0d16e3a0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:02.191529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:02.192154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:02.192436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:02.192713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:02.202093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:02.203582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d4fc8-3ba8-4014-be34-068686750e79 service glance] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:02.213158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1078/2167] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:09.070597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:09.071088 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:09.071182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:09.078648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:09.092969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] MFA Rules not processed for user `aff89a63c86f4e6f906988791f3c5258`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:09.094420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:09.153251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1090/2168] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:12.208948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=13 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:19.146419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=24 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:20.247997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:20.248531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:20.248531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:20.268531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:20.269618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:20.364728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1079/2169] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:14:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:20.413602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:20.415625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:20.497877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f89d9-d213-43b9-af86-26c4722091d3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:20.503073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:20.503662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:20.503943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:20.504211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:20.512709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:20.514359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290a2f1-177c-48dd-8025-ba601da57410 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:20.522853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1091/2170] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:14:30.139985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:30.147680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:30.270071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a19f2c-fbe2-4407-9305-d37303805a0f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:30.276176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:30.277025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:30.277494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:30.277991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:30.280155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:30.292785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:30.294712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:30.306932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1080/2171] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:30.519725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=32 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:37.885224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:37.885828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:37.885828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:37.906636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:37.915765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:37.923718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1092/2172] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:37.937834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:37.949582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db568e21-27b8-4f2a-b176-a9ef399bf477 service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:37.955190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:37.955809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:37.956094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:37.956372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:37.964780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:37.966426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:37.974608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1081/2173] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:14:45.178220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.181130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.258579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16e89f-9027-4bc7-bdae-8eac09baa752 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.264009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.264671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.264940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.265289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] PATH_INFO: `/v3/users/99c077c212f1472d85df56c05a51ceb9` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.266534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c077c212f1472d85df56c05a51ceb9)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.268450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.314974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c077c212f1472d85df56c05a51ceb9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.327038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c077c212f1472d85df56c05a51ceb9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.333902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c077c212f1472d85df56c05a51ceb9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.334580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c077c212f1472d85df56c05a51ceb9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.335190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c077c212f1472d85df56c05a51ceb9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.347799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1093/2174] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:14:45 2022] DELETE /identity/v3/users/99c077c212f1472d85df56c05a51ceb9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:45.384465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.390136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=64 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:45.396340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535ab8-d84f-4666-ac3a-a00acbb68c22 service swift] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.453545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.454288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.454572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.454847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] PATH_INFO: `/v3/users/61d55322f9e348baa681004bc5ee9ccb` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.456082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d55322f9e348baa681004bc5ee9ccb)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.458073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.521152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d55322f9e348baa681004bc5ee9ccb'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.521827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d55322f9e348baa681004bc5ee9ccb'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.529792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.531343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.538860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d55322f9e348baa681004bc5ee9ccb'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.548942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a58d8-7aa9-4029-b0e1-825a504e290c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.550075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d55322f9e348baa681004bc5ee9ccb'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.554390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.555083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.555420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.555710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] PATH_INFO: `/v3/projects/707034727c6c44eb8cea2f5951ac48b7` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.557044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=707034727c6c44eb8cea2f5951ac48b7)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.559061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.560224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-67b7cdfa-96f9-421e-ac1a-3fbf85061431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d55322f9e348baa681004bc5ee9ccb'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.568476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1082/2175] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:14:45 2022] DELETE /identity/v3/users/61d55322f9e348baa681004bc5ee9ccb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:45.574199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.574592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.574876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.596991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:45.598449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.621461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707034727c6c44eb8cea2f5951ac48b7', 'request_id': 'req-bc6729bd-3d21-4a33-a659-e8e94f420dd1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.630614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707034727c6c44eb8cea2f5951ac48b7', 'request_id': 'req-bc6729bd-3d21-4a33-a659-e8e94f420dd1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.637757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707034727c6c44eb8cea2f5951ac48b7', 'request_id': 'req-bc6729bd-3d21-4a33-a659-e8e94f420dd1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.642444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:45.651257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] The token cache is being invalidate because project 707034727c6c44eb8cea2f5951ac48b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:14:45.651559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.652683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bc6729bd-3d21-4a33-a659-e8e94f420dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:45.654431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1094/2176] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:14:45 2022] DELETE /identity/v3/projects/707034727c6c44eb8cea2f5951ac48b7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:45.659227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e212-61ba-450b-8bc6-0e96c1d65fd0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.659515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e212-61ba-450b-8bc6-0e96c1d65fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.659789 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e212-61ba-450b-8bc6-0e96c1d65fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.664535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1083/2177] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:45.674023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.675457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.682039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-77a5e212-61ba-450b-8bc6-0e96c1d65fd0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:45.683223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-77a5e212-61ba-450b-8bc6-0e96c1d65fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.701304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:45.735330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5331d3b-8009-4c11-a53b-2b4a4b12e68f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.749039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1095/2178] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:45.750640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.750929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.751208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.772834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:45.774663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:45.782558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1096/2179] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:45.786652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.786884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.787511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.787796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.788076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.795435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f90fb-2a9e-42b8-bbd7-8dd3ab018b93 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.798091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.799535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.800336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.800936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.801216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.801488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.802214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:14:45.802520 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.804107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.809316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1084/2180] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:14:45.823281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1097/2181] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:14:45.824522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.834697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11cf5-cce9-4fb6-ad57-4f957fe5e8b2 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.840079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.840695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.841011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.841293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.842311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.843908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.875427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1085/2182] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:45.876642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.888227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c11e7-c665-49b3-8c09-f9a98ccb5cda admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:45.952779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:45.953413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:45.953695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:45.953992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:45.954659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:45.956602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:45.981889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1098/2183] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:14:45 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:45.986236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:45.998839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d7027-c3ce-4b84-9ce1-a701c39e9a0e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.003707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.004339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.004622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.004920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.005627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.007489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.022119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1086/2184] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:46.025766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.037138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94a8e-b90e-4ce8-8014-cd5a54338c0a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.042173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.042914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.043534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.043890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.044665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.046745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.062074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1099/2185] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:46.066167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.077962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c4e7a-9e62-467b-b0ee-20160b29b58b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.083676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.084352 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.084635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.084912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] PATH_INFO: `/v3/projects/281db54643a44abb906ad1982ce19e44/users/4bd1b765aaf448048a79084df3535da6/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.102901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281db54643a44abb906ad1982ce19e44, user_id=4bd1b765aaf448048a79084df3535da6, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.105803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f003e-d3da-4f40-ad80-284d5e12de55 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.116660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.119282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.123398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1087/2186] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:46 2022] PUT /identity/v3/projects/281db54643a44abb906ad1982ce19e44/users/4bd1b765aaf448048a79084df3535da6/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:46.155280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd0612-7c60-4e5a-84be-b3ad8b1d8517 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.198079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.198366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.198654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.220998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:46.227066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] MFA Rules not processed for user `4bd1b765aaf448048a79084df3535da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:46.228301 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.229362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.230259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.230543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.230819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] PATH_INFO: `/v3/projects/05e1761d372a41a78c201d95ef0909f5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.232013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e1761d372a41a78c201d95ef0909f5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.233953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.288668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e1761d372a41a78c201d95ef0909f5', 'request_id': 'req-bf426c3e-5119-4152-8017-aa0cd00cedaf'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:46.298394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e1761d372a41a78c201d95ef0909f5', 'request_id': 'req-bf426c3e-5119-4152-8017-aa0cd00cedaf'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:46.301414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1088/2187] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:46.310949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e1761d372a41a78c201d95ef0909f5', 'request_id': 'req-bf426c3e-5119-4152-8017-aa0cd00cedaf'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:46.314915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.316377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.323440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] The token cache is being invalidate because project 05e1761d372a41a78c201d95ef0909f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:14:46.323829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:46.324567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-bf426c3e-5119-4152-8017-aa0cd00cedaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:14:46.329675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1100/2188] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:14:46 2022] DELETE /identity/v3/projects/05e1761d372a41a78c201d95ef0909f5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:46.331185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2cea-d712-432e-86ab-1b7deccc53ae None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.331550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2cea-d712-432e-86ab-1b7deccc53ae None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.331885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2cea-d712-432e-86ab-1b7deccc53ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.337634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:46.353050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-12ec2cea-d712-432e-86ab-1b7deccc53ae None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:46.354698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-12ec2cea-d712-432e-86ab-1b7deccc53ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.390844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa325-a968-44ef-becb-67db8132d2b2 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.425884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1101/2189] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:46.431203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.431467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.431622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.455121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.455740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.456025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.456308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.457054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:46.458129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.458300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.466508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1102/2190] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:14:46.470826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.480579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dfb56-79db-45d3-8643-6d521af992a6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.485527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.486179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.487109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.487394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.488093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:14:46.488408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.490247 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.508082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1103/2191] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:14:46.509131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.518918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2c68b-ef65-4987-8ead-8a0887186cfc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.523736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.524099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.524341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.524622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.524897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.525557 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.525718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.527480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.535359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1089/2192] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:46.559351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1104/2193] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:14:46.560464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.571146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651b07-ec93-4b22-ac00-da66d267b38a service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.627891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.628488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.628769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.629047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.629707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.631671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.655861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1090/2194] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:46.657111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.669117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fea83-d6fd-4cff-8196-461a5a4f63a7 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.674976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.675629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.675910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.676182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.676882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.678694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.693921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1105/2195] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:46.698144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.710362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12944-7179-49e5-9a9b-208293f24ea4 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.717500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.718158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.718453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.718764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.719455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.721447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04775045-5ed8-4c9f-b4a2-2b1cdea35128 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.734175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1091/2196] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:46.738524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.749867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edd3fc-4172-4cae-b393-f98baa4b5603 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:46.756284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:46.757016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.757311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.757687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] PATH_INFO: `/v3/projects/84acd1145d1c4c7db932f7f104ad8770/users/2a8102e28f574804b5655d8f99dedfb1/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.777347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84acd1145d1c4c7db932f7f104ad8770, user_id=2a8102e28f574804b5655d8f99dedfb1, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:46.780149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:46.807104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1106/2197] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:46 2022] PUT /identity/v3/projects/84acd1145d1c4c7db932f7f104ad8770/users/2a8102e28f574804b5655d8f99dedfb1/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:46.817565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:46.817882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:46.818165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:46.828996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:46.859232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] MFA Rules not processed for user `2a8102e28f574804b5655d8f99dedfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:46.860410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:46.945402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1092/2198] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:46.949903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:46.960458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:47.014708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37696a-0674-4563-a9d8-a2ede74e96b4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.019427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.020056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.020337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.020616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.021280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.023744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.050766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1107/2199] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:14:46 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:14:47.054800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.065536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4c7c4-53ff-4644-9ec4-66d849c14344 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.121753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.122410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.122692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.123307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.123638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.125567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.147849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1093/2200] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:14:47 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:14:47.152576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.164190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ea5b-7852-4154-bd24-da62ecef06ac admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.176245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.176902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.177196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.177483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.178239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.179933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.191064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1108/2201] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:47.195636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.207872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ab8b9-86d6-457d-9041-ae28679ffd62 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.213139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.213751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.214068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.214346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.215041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.216633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cd5fc-61ed-4548-8c25-e30ac76f0f1e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.226793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1094/2202] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:47.231030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.241584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfe3c0-4105-4a23-9208-a6c232b5d87d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.247736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.248417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.248689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.248968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] PATH_INFO: `/v3/projects/9cbaa8561005437885235bcf07af988a/users/78c396ec537e4008b395c1cb5b6d2339/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.264438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbaa8561005437885235bcf07af988a, user_id=78c396ec537e4008b395c1cb5b6d2339, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.267207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.282050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1109/2203] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:47 2022] PUT /identity/v3/projects/9cbaa8561005437885235bcf07af988a/users/78c396ec537e4008b395c1cb5b6d2339/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:47.352581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.352813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.353060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.380159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] MFA Rules not processed for user `78c396ec537e4008b395c1cb5b6d2339`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:47.381470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:47.444152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:47.449263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1095/2204] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:47.454431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.529804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70134766-05bf-416b-8075-ddee6e5afc67 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.534804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.535450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.535722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.535993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.536660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.538650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.567316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1110/2205] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:14:47 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:47.571999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.583143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314429a-1fbb-4a71-b01b-ad5b9a73230c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.641432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.642094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.642379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.642659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.643339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.645313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.666170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1096/2206] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:14:47 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:14:47.673728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.684867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729153a-4096-4327-baad-d837e38956c7 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.689598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.690249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.690535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.690811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.691575 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.693279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.709301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1111/2207] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:47.710521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.721023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139eade-60b8-4d38-a9df-12da43c6d118 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.725859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.726535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.726816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.727088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] PATH_INFO: `/v3/projects/98989b0c094541e1805ba68079e5ed01/users/0547d5b883d44af8a26cd741f99ba8e3/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.743640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98989b0c094541e1805ba68079e5ed01, user_id=0547d5b883d44af8a26cd741f99ba8e3, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.746389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.761731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1097/2208] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:47 2022] PUT /identity/v3/projects/98989b0c094541e1805ba68079e5ed01/users/0547d5b883d44af8a26cd741f99ba8e3/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:47.765737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.775855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f117e-c977-4a8c-b654-b8bfdf3b9dc4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.807587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:47.835093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.835739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.836027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.836302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.836995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.838629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8ae5d-2fb6-42f6-9723-3578ce5f714b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.848248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1112/2209] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:14:47.853278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:47.863538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6dfa6-1d42-498e-b700-595e087af1c2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:47.870884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:47.871729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.872186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.872607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] PATH_INFO: `/v3/projects/98989b0c094541e1805ba68079e5ed01/users/0547d5b883d44af8a26cd741f99ba8e3/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.874931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98989b0c094541e1805ba68079e5ed01, user_id=0547d5b883d44af8a26cd741f99ba8e3, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:47.877858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:47.888692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1098/2210] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:14:47 2022] PUT /identity/v3/projects/98989b0c094541e1805ba68079e5ed01/users/0547d5b883d44af8a26cd741f99ba8e3/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:14:47.898933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:47.899220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:47.899497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:47.931151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] MFA Rules not processed for user `0547d5b883d44af8a26cd741f99ba8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:14:47.932427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:48.005320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1113/2211] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:14:48.016673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:48.018505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:48.081777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ab51b-d880-4c54-aa83-fcce9f50da10 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:48.085551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:48.086307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:48.086593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:48.086870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:48.144130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:48.145567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:48.154612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1099/2212] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:50.783169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:50.785263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:14:50.809495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c1c07-0ee5-46d2-b0d4-725fa376c175 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:50.833599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:50.834846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:50.835135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:50.835521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:50.909176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:50.910781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:50.921191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1114/2213] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:14:51.519776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:14:51.528927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c7d3-e4ee-4876-98f6-f713b07cdf8f service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:14:51.532455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:14:51.533062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:14:51.533346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:14:51.533618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:14:51.540304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:14:51.544838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:14:51.547003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59778e0c-903e-4df8-ad6b-102ef4176e4f service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:14:51.557906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1100/2214] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:14:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:15:00.916312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=76 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:01.553025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:04.914426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:04.979113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ccdc0-47e7-479b-a2a0-9ca45a9b2f6a service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:04.983607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:04.984246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:04.984528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:04.984805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] PATH_INFO: `/v3/users/2a8102e28f574804b5655d8f99dedfb1` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:04.985993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8102e28f574804b5655d8f99dedfb1)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:04.987911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:05.034231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8102e28f574804b5655d8f99dedfb1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.046489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8102e28f574804b5655d8f99dedfb1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.053143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8102e28f574804b5655d8f99dedfb1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.053811 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8102e28f574804b5655d8f99dedfb1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.054501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8102e28f574804b5655d8f99dedfb1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.067212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1115/2215] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:15:04 2022] DELETE /identity/v3/users/2a8102e28f574804b5655d8f99dedfb1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:05.072254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:05.072774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:05.072869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:05.092529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:15:05.094117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:05.153263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1101/2216] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:15:05.163437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:05.164895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:05.223039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd790d-25ca-4c11-8720-4c5462b3ede1 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:05.227717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:05.228497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:05.228778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:05.229221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:05.244390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:05.245953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:05.256272 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1116/2217] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:15:05.536492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:05.538180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:05.552272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ce399-694b-4cae-92d9-0497e8cc21f7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:05.559609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:05.560190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:05.560476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:05.560757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] PATH_INFO: `/v3/projects/84acd1145d1c4c7db932f7f104ad8770` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:05.562086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84acd1145d1c4c7db932f7f104ad8770)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:05.572999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:05.654836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84acd1145d1c4c7db932f7f104ad8770', 'request_id': 'req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.662928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84acd1145d1c4c7db932f7f104ad8770', 'request_id': 'req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.671793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84acd1145d1c4c7db932f7f104ad8770', 'request_id': 'req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.685211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] The token cache is being invalidate because project 84acd1145d1c4c7db932f7f104ad8770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:15:05.685519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.686423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.690848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1102/2218] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:15:05 2022] DELETE /identity/v3/projects/84acd1145d1c4c7db932f7f104ad8770 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:05.692327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:05.707510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756197d8-b07f-4f76-b77c-3d21902a6229 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:05.729202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:05.771387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:05.772035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:05.772316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:05.772593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] PATH_INFO: `/v3/users/78c396ec537e4008b395c1cb5b6d2339` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:05.773786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c396ec537e4008b395c1cb5b6d2339)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:05.775794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:05.821039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c396ec537e4008b395c1cb5b6d2339'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.833334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c396ec537e4008b395c1cb5b6d2339'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.840227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c396ec537e4008b395c1cb5b6d2339'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.840894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c396ec537e4008b395c1cb5b6d2339'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.841495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c396ec537e4008b395c1cb5b6d2339'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:05.855070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1117/2219] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:15:05 2022] DELETE /identity/v3/users/78c396ec537e4008b395c1cb5b6d2339 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:06.134906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:06.136443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:06.151727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5b4-3bda-46cf-8dc1-2be307735eb7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:06.217110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:06.217763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:06.218095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:06.218374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] PATH_INFO: `/v3/projects/9cbaa8561005437885235bcf07af988a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:06.219593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cbaa8561005437885235bcf07af988a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:06.221950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:06.257189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:06.278325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cbaa8561005437885235bcf07af988a', 'request_id': 'req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.285906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cbaa8561005437885235bcf07af988a', 'request_id': 'req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.293420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cbaa8561005437885235bcf07af988a', 'request_id': 'req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.306093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] The token cache is being invalidate because project 9cbaa8561005437885235bcf07af988a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:15:06.306383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.307039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.308559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1103/2220] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:15:06 2022] DELETE /identity/v3/projects/9cbaa8561005437885235bcf07af988a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:06.312710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:06.314173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:06.330211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5387760-0524-4252-aed9-d4bdbaed2bac admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:06.396075 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:06.396755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:06.397018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:06.397291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] PATH_INFO: `/v3/users/0547d5b883d44af8a26cd741f99ba8e3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:06.399026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0547d5b883d44af8a26cd741f99ba8e3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:06.401055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:06.446516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:06.449631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0547d5b883d44af8a26cd741f99ba8e3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.465363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0547d5b883d44af8a26cd741f99ba8e3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.472418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0547d5b883d44af8a26cd741f99ba8e3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.473129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0547d5b883d44af8a26cd741f99ba8e3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.473727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-677456de-992a-44a6-aafc-f08993816e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0547d5b883d44af8a26cd741f99ba8e3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.488531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1118/2221] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:15:06 2022] DELETE /identity/v3/users/0547d5b883d44af8a26cd741f99ba8e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:06.777899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:06.777899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:06.791505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd4712-8c2a-44b2-85f2-3f2ac3ba827c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:06.865647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:06.866415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:06.866699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:06.867036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] PATH_INFO: `/v3/projects/98989b0c094541e1805ba68079e5ed01` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:06.868209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98989b0c094541e1805ba68079e5ed01)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:06.870154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:06.917053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98989b0c094541e1805ba68079e5ed01', 'request_id': 'req-bfe429fb-922c-451b-8bff-2ce6b4507a3d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.925174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98989b0c094541e1805ba68079e5ed01', 'request_id': 'req-bfe429fb-922c-451b-8bff-2ce6b4507a3d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.932256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98989b0c094541e1805ba68079e5ed01', 'request_id': 'req-bfe429fb-922c-451b-8bff-2ce6b4507a3d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.942729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] The token cache is being invalidate because project 98989b0c094541e1805ba68079e5ed01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:15:06.943024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.943689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bfe429fb-922c-451b-8bff-2ce6b4507a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:15:06.945738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1104/2222] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:15:06 2022] DELETE /identity/v3/projects/98989b0c094541e1805ba68079e5ed01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:06.951072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:06.951362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:06.951638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:06.972878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:15:06.974118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:07.031628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1119/2223] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:15:07.039641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.039930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.040211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.060948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:15:07.062309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:07.071124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1105/2224] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:15:07.075298 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.086099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9bf0-5c31-46cb-93af-a67da4ac3ab8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.090666 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.091273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.091548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.091819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.092564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:15:07.092813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.094515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.105918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1120/2225] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:15:07.110319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.120367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911242a-308e-49a8-a0ef-acda77dd9369 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.125103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.125713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.126030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.126326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.126969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.128904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.155173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1106/2226] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:15:07 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:15:07.159455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.169399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010561f0-1e98-4c5d-b6c6-cb7cfc0e56d0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.224136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.224751 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.225030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.225303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.226052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.228027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.249942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1121/2227] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:15:07 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:15:07.254040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.265068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce85b6-efcd-4350-b96f-ed5de702ebd7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.269662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.270302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.270583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.270856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.271545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.273156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.283376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1107/2228] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:15:07.287109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.296938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0107eb7-1120-4896-af8e-df3fb230c502 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.301685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.302393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.302675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.302949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.303644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.305737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb789c-3782-49f3-b60f-d2205665bd55 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.316335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1122/2229] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:15:07.320934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.327415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:07.331135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9106a-c4e4-4e8e-87a4-ae4cf5927371 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.337126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.337984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.338472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.338941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] PATH_INFO: `/v3/projects/afe2b5c8e5d9448da362b23ce6614b59/users/41c4d96cd9c34b4e978eb79eb317a0c0/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.356567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe2b5c8e5d9448da362b23ce6614b59, user_id=41c4d96cd9c34b4e978eb79eb317a0c0, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.359439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.372878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1108/2230] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:15:07 2022] PUT /identity/v3/projects/afe2b5c8e5d9448da362b23ce6614b59/users/41c4d96cd9c34b4e978eb79eb317a0c0/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:15:07.384062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84b319-a4a9-4308-8516-2ae6b5aa98f8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.384348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84b319-a4a9-4308-8516-2ae6b5aa98f8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.384624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84b319-a4a9-4308-8516-2ae6b5aa98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.411724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-df84b319-a4a9-4308-8516-2ae6b5aa98f8 None None] MFA Rules not processed for user `41c4d96cd9c34b4e978eb79eb317a0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:15:07.412916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-df84b319-a4a9-4308-8516-2ae6b5aa98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:07.425533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:07.479719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1123/2231] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:15:07.490995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:15:07.493244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:07.547729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad16fe-2595-41bf-8c2f-756f2be8d67b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:15:07.551332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:15:07.551941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:07.552218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:07.552489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:07.561381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:15:07.562914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:15:07.571759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1109/2232] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:15:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:15:11.892951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097619ea-3b76-4888-89d9-00d216a08e9e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:15:11.893449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097619ea-3b76-4888-89d9-00d216a08e9e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:15:11.893541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097619ea-3b76-4888-89d9-00d216a08e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:15:11.912216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-097619ea-3b76-4888-89d9-00d216a08e9e None None] MFA Rules not processed for user `79e4add99b984c25b04553927bed7cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:15:11.913416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-097619ea-3b76-4888-89d9-00d216a08e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:15:11.973861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1124/2233] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:15:17.567953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:15:21.967102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=45 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:15.574965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:15.576474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:15.648564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bd2b9-0722-424b-8ce1-358f87ec3321 service glance] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:15.656733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:15.657528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:15.658126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:15.658596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] PATH_INFO: `/v3/users/4bd1b765aaf448048a79084df3535da6` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:15.659988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd1b765aaf448048a79084df3535da6)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:15.662448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:15.713453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd1b765aaf448048a79084df3535da6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:15.714153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd1b765aaf448048a79084df3535da6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:15.727643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd1b765aaf448048a79084df3535da6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:15.734881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd1b765aaf448048a79084df3535da6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:15.741884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd1b765aaf448048a79084df3535da6'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:15.747924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1110/2234] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:16:15 2022] DELETE /identity/v3/users/4bd1b765aaf448048a79084df3535da6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:15.752894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:15.753386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:15.753497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:15.773479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:15.774973 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:15.839048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1125/2235] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:15.848630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:15.850066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:15.907999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9f32b-ee91-4499-8f81-9e64278bb20f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:15.911711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:15.912326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:15.912603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:15.912875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:15.923158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:15.925202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:15.940402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1111/2236] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:16.488308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:16.490306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:16.505144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:16.506938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:16.526333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d3522-9ffb-445f-a053-c2d6d5ef9d4a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:16.531278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6f050-f552-444b-b9a6-1c8d6a152ed9 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:16.533703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:16.536032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:16.536347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:16.536664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] PATH_INFO: `/v3/projects/281db54643a44abb906ad1982ce19e44` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:16.537818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:16.538628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:16.538915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:16.539020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281db54643a44abb906ad1982ce19e44)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:16.539190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] PATH_INFO: `/v3/users/41c4d96cd9c34b4e978eb79eb317a0c0` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:16.540502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c4d96cd9c34b4e978eb79eb317a0c0)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:16.541426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:16.542523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:16.564318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:16.603314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281db54643a44abb906ad1982ce19e44', 'request_id': 'req-50cd61b3-25ae-4384-997f-fb11a8127e61'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.606561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c4d96cd9c34b4e978eb79eb317a0c0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.607404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c4d96cd9c34b4e978eb79eb317a0c0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.611048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281db54643a44abb906ad1982ce19e44', 'request_id': 'req-50cd61b3-25ae-4384-997f-fb11a8127e61'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.619165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281db54643a44abb906ad1982ce19e44', 'request_id': 'req-50cd61b3-25ae-4384-997f-fb11a8127e61'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.620900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c4d96cd9c34b4e978eb79eb317a0c0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.630622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c4d96cd9c34b4e978eb79eb317a0c0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.632108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] The token cache is being invalidate because project 281db54643a44abb906ad1982ce19e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:16:16.632533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.633501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-50cd61b3-25ae-4384-997f-fb11a8127e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.643711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1126/2237] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:16:16 2022] DELETE /identity/v3/projects/281db54643a44abb906ad1982ce19e44 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:16.656824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c4d96cd9c34b4e978eb79eb317a0c0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:16.666101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1112/2238] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:16:16 2022] DELETE /identity/v3/users/41c4d96cd9c34b4e978eb79eb317a0c0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:16.671362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:16.671875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:16.672176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:16.696673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:16.697950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:16.792525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1127/2239] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:16.803042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:16.805061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:16.871409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df641fd8-c234-4a78-b48d-5e10e81cec17 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:16.875629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:16.876280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:16.876564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:16.876845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:16.885708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:16.887180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bca09-183f-45f6-b210-53a4487ffb3b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:16.896620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1113/2240] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:17.164834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.166289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:17.179856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf817397-3fed-4507-8e0d-493d44c1fdad None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.184623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.185769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.186090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.186534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] PATH_INFO: `/v3/projects/afe2b5c8e5d9448da362b23ce6614b59` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.188317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe2b5c8e5d9448da362b23ce6614b59)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.190671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.211720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:17.237737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe2b5c8e5d9448da362b23ce6614b59', 'request_id': 'req-3eafc937-49a6-4c8c-a5e9-de3e678ea272'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:17.244912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe2b5c8e5d9448da362b23ce6614b59', 'request_id': 'req-3eafc937-49a6-4c8c-a5e9-de3e678ea272'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:17.251208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe2b5c8e5d9448da362b23ce6614b59', 'request_id': 'req-3eafc937-49a6-4c8c-a5e9-de3e678ea272'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:17.261481 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] The token cache is being invalidate because project afe2b5c8e5d9448da362b23ce6614b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:16:17.261770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:17.262427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3eafc937-49a6-4c8c-a5e9-de3e678ea272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:17.263930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1128/2241] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:16:17 2022] DELETE /identity/v3/projects/afe2b5c8e5d9448da362b23ce6614b59 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:17.268759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992407d-e940-4c45-b274-7e151affa67b None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.269070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992407d-e940-4c45-b274-7e151affa67b None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.269343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992407d-e940-4c45-b274-7e151affa67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.288921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c992407d-e940-4c45-b274-7e151affa67b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:17.290128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c992407d-e940-4c45-b274-7e151affa67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:17.346577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1114/2242] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:17.358625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.358931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.359823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.379219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:17.380423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:17.388342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1129/2243] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:17.392662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992407d-e940-4c45-b274-7e151affa67b None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.402285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992407d-e940-4c45-b274-7e151affa67b None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.407389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.408104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.408401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.408731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.409462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:16:17.409802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.411617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.423607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1115/2244] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:16:17.427721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.437002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0dcd9-1182-4c87-96ff-e201cabd4812 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.442650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.443274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.443553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.443824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.444548 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.446739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.475017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1130/2245] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:16:17 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:16:17.479613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.490365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d90c-7a47-45d2-81a8-aa6ced43a6ac admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.514837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:17.554153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.554775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.555096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.555327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.555987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.557911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.579469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1116/2246] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:16:17 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:16:17.583614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.593866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a983f-0743-4b1e-8ccf-4d1a3e64bb6c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.598427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.599034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.599315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.599588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.600283 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.601897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.612166 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1131/2247] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:16:17.615950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.626441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6248f1-32ba-4100-a120-e846f66e74ea admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.630929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.631529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.631806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.632075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.632756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.634370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ff73-419a-41b5-a48e-5b0dedcafe5f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.643987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1117/2248] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:16:17.647968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.657860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8f565-a475-447c-a730-446cc45d7273 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.662785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.663382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.663662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.663935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] PATH_INFO: `/v3/projects/a81087cbb6344ae7b09f15aa64c7dfa8/users/b9b4a63005384b7ab2601747e1e67dba/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.678583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81087cbb6344ae7b09f15aa64c7dfa8, user_id=b9b4a63005384b7ab2601747e1e67dba, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.681210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.693697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1132/2249] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:16:17 2022] PUT /identity/v3/projects/a81087cbb6344ae7b09f15aa64c7dfa8/users/b9b4a63005384b7ab2601747e1e67dba/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:16:17.704360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.704649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.704928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.731404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] MFA Rules not processed for user `b9b4a63005384b7ab2601747e1e67dba`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:17.732607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:17.795842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1118/2250] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:17.808511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:17.810268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:17.867150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae586-bba4-4677-a99a-93a0ebbe43dd admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:17.871019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:17.871631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:17.871909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:17.872183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:17.880832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:17.882285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:17.891210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1133/2251] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:19.565217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:19.565217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:19.621767 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a54ab-791b-4433-94cf-43d5e8d63761 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:19.625641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:19.626245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:19.626523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:19.626808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:19.629827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:19.675714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:19.702222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:19.704211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:19.714932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1119/2252] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:19.872301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:19.873694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:19.925508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:19.931086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7691-fd08-4a68-8fd2-1b2d1fe3d1d4 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:19.935371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:19.936351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:19.936734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:19.936895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:19.939514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:19.952906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:19.954516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:19.969732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1134/2253] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:28.106595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:28.109605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:28.124555 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709201bc-8173-4b94-9d19-ccc248c1eda6 service glance] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:28.129416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:28.130297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:28.130788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:28.131158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:28.134876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:28.211770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:28.213447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:28.226004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1120/2254] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:29.506546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:29.515953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8807-e930-4763-a41f-3f485c6c715b service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:29.519677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:29.520364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:29.520717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:29.521101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:29.523142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:29.582514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:29.584119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:29.594919 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1135/2255] 10.222.0.48 () {60 vars in 1396 bytes} [Thu Jul 28 09:16:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:30.514030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:30.523614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85f44e-6add-4565-afd6-71a2199d4b1d service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:30.527198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:30.527801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:30.528079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:30.528351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:30.530251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:30.590478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:30.591915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3788fc-920d-40ef-82c8-d8a325aae9d1 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:30.601528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1121/2256] 10.222.0.48 () {60 vars in 1396 bytes} [Thu Jul 28 09:16:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:33.415365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:33.477210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c9f2-2523-4ea2-b48f-775717ad69a5 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:33.481653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:33.482325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:33.482607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:33.482883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] PATH_INFO: `/v3/users/b9b4a63005384b7ab2601747e1e67dba` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:33.484555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b4a63005384b7ab2601747e1e67dba)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:33.486576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:33.531467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b4a63005384b7ab2601747e1e67dba'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:33.542985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b4a63005384b7ab2601747e1e67dba'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:33.550214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b4a63005384b7ab2601747e1e67dba'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:33.550876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b4a63005384b7ab2601747e1e67dba'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:33.551651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b4a63005384b7ab2601747e1e67dba'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:33.565339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1136/2257] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:16:33 2022] DELETE /identity/v3/users/b9b4a63005384b7ab2601747e1e67dba => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:33.569896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:33.570379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:33.570481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:33.589191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:33.590416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:33.647190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1122/2258] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:33.656550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:33.658013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:33.669132 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:33.718788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53237bf3-4990-4807-a53a-51cdfe30d944 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:33.722426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:33.723051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:33.723329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:33.723603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:33.732384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:33.733826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:33.743621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1137/2259] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:34.163887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:34.165683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:34.180678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a0751-9061-4b9b-997b-3a5af0bc7d25 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:34.185398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:34.186076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:34.186368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:34.186648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] PATH_INFO: `/v3/projects/a81087cbb6344ae7b09f15aa64c7dfa8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:34.188297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81087cbb6344ae7b09f15aa64c7dfa8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:34.189935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:34.197417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:34.265827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81087cbb6344ae7b09f15aa64c7dfa8', 'request_id': 'req-eb93d23f-eb0e-4d51-8355-8dd24264e63f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.279491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81087cbb6344ae7b09f15aa64c7dfa8', 'request_id': 'req-eb93d23f-eb0e-4d51-8355-8dd24264e63f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.288356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81087cbb6344ae7b09f15aa64c7dfa8', 'request_id': 'req-eb93d23f-eb0e-4d51-8355-8dd24264e63f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.302531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] The token cache is being invalidate because project a81087cbb6344ae7b09f15aa64c7dfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:16:34.302927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.303593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.306493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1123/2260] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:16:34 2022] DELETE /identity/v3/projects/a81087cbb6344ae7b09f15aa64c7dfa8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:34.621723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:34.623301 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:34.637623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050ad1e-978e-41f9-92b7-d962d3933f05 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:34.697780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:34.698465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:34.698742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:34.699013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] PATH_INFO: `/v3/users/79e4add99b984c25b04553927bed7cb5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:34.700158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e4add99b984c25b04553927bed7cb5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:34.702397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:34.748078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e4add99b984c25b04553927bed7cb5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.760151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e4add99b984c25b04553927bed7cb5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.767492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e4add99b984c25b04553927bed7cb5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.768159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e4add99b984c25b04553927bed7cb5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.768753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-5ad8d4f7-0871-466c-95f5-36d26a007550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e4add99b984c25b04553927bed7cb5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:34.781959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1138/2261] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:16:34 2022] DELETE /identity/v3/users/79e4add99b984c25b04553927bed7cb5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:34.958198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:34.959718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:34.973023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d23f-eb0e-4d51-8355-8dd24264e63f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:35.030315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:35.030989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:35.031275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:35.031551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] PATH_INFO: `/v3/projects/39a7fbb2f3ca405bbaa394825b4a3e73` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:35.032708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a7fbb2f3ca405bbaa394825b4a3e73)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:35.034647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:35.080849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a7fbb2f3ca405bbaa394825b4a3e73', 'request_id': 'req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:35.087483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a7fbb2f3ca405bbaa394825b4a3e73', 'request_id': 'req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:35.096838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a7fbb2f3ca405bbaa394825b4a3e73', 'request_id': 'req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:35.104522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] The token cache is being invalidate because project 39a7fbb2f3ca405bbaa394825b4a3e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:16:35.104853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:35.105510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:16:35.107070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1124/2262] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:16:34 2022] DELETE /identity/v3/projects/39a7fbb2f3ca405bbaa394825b4a3e73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:16:37.551440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:37.551982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:37.551982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:37.571052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] MFA Rules not processed for user `e0c1ed47784c46baa7ddc4abdfc54bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:37.572255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:37.595908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:37.627442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1139/2263] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:16:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:38.008576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:38.010025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:38.040394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:38.071235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696ab8-f31c-4dd6-9f60-cbe87183cef2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:38.074866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:38.075491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:38.075770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:38.076039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:38.078644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:38.088419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:38.089857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:38.098868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1125/2264] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:38.438345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:38.439729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:38.515068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592742c-6524-4af3-bf9d-3567e56c93b8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:38.520596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:38.521210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:38.521488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:38.521761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:38.523524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:38.534922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:38.536485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efa1ba-549f-4aab-a0ff-9d77fdb29ca0 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:38.546152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1140/2265] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:41.471848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:41.481516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5d3ef-a216-440d-a00c-3762bdf8df01 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:41.486495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:41.487183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:41.487527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:41.487865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:41.489640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:41.501614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:41.503670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:41.511519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1126/2266] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:48.542082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=44 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:16:49.253778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:49.254103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:49.254411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:49.273386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:16:49.274633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:49.331289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1141/2267] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:16:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:16:49.379108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:49.380508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:49.446898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973289b-39d5-4c5c-becd-0c9f74cb4b9b service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:49.451740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:49.452616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:49.453020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:49.453297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:49.461413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:49.463064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:49.472273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1127/2268] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:16:56.749592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:16:56.751855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:56.844226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8bc6-e904-4318-a487-8d6a332384f4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:16:56.852719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:16:56.853633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:16:56.854369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:16:56.854713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:16:56.856606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:16:56.879271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:16:56.881659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c64a5-6608-42b5-be8b-07c9361db7f1 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:16:56.896594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1142/2269] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:16:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:16:59.468090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=53 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:17:03.024447 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:03.033544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a3bc6-ef69-4837-b152-8e3f31b8e2f1 service swift] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:03.037068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:03.037694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.038001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.038276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.039991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.049274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:03.050724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b94e9-e9ca-4ee7-a19c-75812681d893 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:03.058492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1128/2270] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:03.179005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.179700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.180004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.181391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1143/2271] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:17:03.187256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0c41-dc4f-4fab-b7db-832e30338b8d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.187538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0c41-dc4f-4fab-b7db-832e30338b8d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.187809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0c41-dc4f-4fab-b7db-832e30338b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.209306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b6bb0c41-dc4f-4fab-b7db-832e30338b8d None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:03.210557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b6bb0c41-dc4f-4fab-b7db-832e30338b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.220012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1129/2272] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:17:03.231783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:03.233509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.295079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c23d-e9c9-47e9-9387-7ed8d31c5bfe None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:03.298583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:03.299242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.299563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.299734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.310149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:03.311637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5364-cc0a-4301-9f95-a1e9cd10e1d1 service glance] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:03.322868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1144/2273] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:03.391128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1146-25f7-489e-b039-db12b6094860 None None] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.391416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1146-25f7-489e-b039-db12b6094860 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.392028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df1146-25f7-489e-b039-db12b6094860 None None] PATH_INFO: `/` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.393428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1130/2274] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:17:03.398260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc99c-6def-4591-8042-543025ea39fd None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.398547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc99c-6def-4591-8042-543025ea39fd None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.398826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc99c-6def-4591-8042-543025ea39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.418223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-63ecc99c-6def-4591-8042-543025ea39fd None None] MFA Rules not processed for user `6077e3f936ea42acbc6879758ddc8336`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:03.419421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-63ecc99c-6def-4591-8042-543025ea39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.426417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1145/2275] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:17:03.437856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df1146-25f7-489e-b039-db12b6094860 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:03.440102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-36df1146-25f7-489e-b039-db12b6094860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.453019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df1146-25f7-489e-b039-db12b6094860 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:03.456477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:03.457073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.457349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.457620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.464402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:03.465811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e4e03-a4ea-4965-b25e-57e04e139629 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:03.473288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1131/2276] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:03.554802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea48a97-18e6-4c4e-9d59-d40fd9c2ef8e None None] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.555093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea48a97-18e6-4c4e-9d59-d40fd9c2ef8e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.555370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea48a97-18e6-4c4e-9d59-d40fd9c2ef8e None None] PATH_INFO: `/` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.556746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1146/2277] 10.222.0.48 () {54 vars in 921 bytes} [Thu Jul 28 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 09:17:03.561679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830abc5e-b287-433a-bdc8-70a80b0a7d2a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.561989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830abc5e-b287-433a-bdc8-70a80b0a7d2a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.562267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830abc5e-b287-433a-bdc8-70a80b0a7d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.581994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-830abc5e-b287-433a-bdc8-70a80b0a7d2a None None] MFA Rules not processed for user `96a7013052fd4d48a2463cf51f13861f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:03.583173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-830abc5e-b287-433a-bdc8-70a80b0a7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:03.592226 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1132/2278] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Jul 28 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:17:03.602936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea48a97-18e6-4c4e-9d59-d40fd9c2ef8e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:03.614002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea48a97-18e6-4c4e-9d59-d40fd9c2ef8e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:03.617497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:03.618130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:03.618411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:03.618683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:03.628591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:03.630035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:03.640351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1147/2279] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:06.521419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:06.521925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:06.522078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:06.540584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:06.541881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:06.548888 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1133/2280] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:17:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:17:06.561072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:06.572853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801a34-5b47-45da-b49c-6aee69c0b439 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:06.578061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:06.578688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:06.578964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:06.579231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:06.587092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:06.588769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:06.593207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=100 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:17:06.597067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1148/2281] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:17:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:17:11.385241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:11.399212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea54a-afe5-49ff-b44e-0df3c03258bc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:11.406579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:11.407218 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:11.407501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:11.407777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:11.410392 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:11.471479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:11.473782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63781f-c926-4769-9ec8-865c6301db9b service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:11.483576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1134/2282] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:21.479123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=49 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:17:32.210230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:32.215211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:32.232196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4d4c1-604b-4238-8d15-6f6b0d46f2d6 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:32.237615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:32.238348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:32.238626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:32.238904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:32.240688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:32.252875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:32.254515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:32.266195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1149/2283] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:17:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:17:34.750258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a4fd-a826-4338-870e-bee687ce2e7a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:34.750740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a4fd-a826-4338-870e-bee687ce2e7a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:34.751091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a4fd-a826-4338-870e-bee687ce2e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:34.770424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7356a4fd-a826-4338-870e-bee687ce2e7a None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:34.771613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7356a4fd-a826-4338-870e-bee687ce2e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:34.778809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1135/2284] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:17:34.790754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:17:34.802024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06645dd5-24f9-4deb-91db-06f7345ffce6 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:17:34.807523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:17:34.808141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:34.808423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:34.808700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:34.816618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:17:34.818153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:17:34.826589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1150/2285] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:17:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:17:44.765535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:17:44.823336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=12 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:17:56.931487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c3322-9d78-438f-8b9a-d0da7ab44007 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:17:56.932009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c3322-9d78-438f-8b9a-d0da7ab44007 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:17:56.932113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c3322-9d78-438f-8b9a-d0da7ab44007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:17:56.950930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bb2c3322-9d78-438f-8b9a-d0da7ab44007 None None] MFA Rules not processed for user `aff89a63c86f4e6f906988791f3c5258`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:17:56.952135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bb2c3322-9d78-438f-8b9a-d0da7ab44007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:17:56.960277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1136/2286] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:17:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:18:03.233554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:18:03.250391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175dd00-23c4-448b-a787-3d88306f9514 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:18:03.256082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:18:03.256697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:18:03.256976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:18:03.257249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:18:03.259054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:18:03.272072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:18:03.273668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b630deb-5e5a-4980-a9b9-c5e0ce00f510 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:18:03.286005 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1151/2287] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:18:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:18:06.946379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:18:13.281066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=6 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:18:38.136881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8ab8-2e46-41b8-837d-3399681542d5 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:18:38.137454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8ab8-2e46-41b8-837d-3399681542d5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:18:38.137454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8ab8-2e46-41b8-837d-3399681542d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:18:38.156097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-235d8ab8-2e46-41b8-837d-3399681542d5 None None] MFA Rules not processed for user `e0c1ed47784c46baa7ddc4abdfc54bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:18:38.157281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-235d8ab8-2e46-41b8-837d-3399681542d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:18:38.163995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1137/2288] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:18:48.151506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:18:49.542085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:18:49.542597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:18:49.542707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:18:49.561279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] MFA Rules not processed for user `a6f5e471a7ac4218a35ffe1840d91575`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:18:49.562494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:18:49.615937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1152/2289] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:18:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:18:59.609329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=25 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:18.746711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:18.747365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:18.747365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:18.774662 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] MFA Rules not processed for user `87b166b84ccd4300a1b862d375c137a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:18.775885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:18.857818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1138/2290] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:28.855845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=37 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:30.549878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:30.610325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821679-70bc-4e7a-871a-c4b323b81650 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:30.615583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:30.616240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:30.616518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:30.616790 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:30.618623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:30.642471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:30.644070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:30.654488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1153/2291] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:31.389081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:31.407893 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059476-66ca-4df7-91a4-7f7867fd7cc9 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:31.411424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:31.412030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:31.412306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:31.412572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:31.414384 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:31.630289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:31.631751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:31.650845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1139/2292] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:35.374900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:35.376348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:35.389606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4757254-c127-4048-8c39-bb00ede9fed3 service placement] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:35.395791 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:35.396423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:35.396709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:35.396987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:35.398776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:35.414198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:35.414198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:35.428287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1154/2293] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:35.428829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:35.430484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:35.442912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb94b5-660f-4dc7-b09b-041de19c2eba service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:35.448233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:35.448964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:35.449247 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:35.449513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] PATH_INFO: `/v3/users/aff89a63c86f4e6f906988791f3c5258` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:35.450890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff89a63c86f4e6f906988791f3c5258)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:35.452976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:35.499582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff89a63c86f4e6f906988791f3c5258'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.500225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff89a63c86f4e6f906988791f3c5258'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.511818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff89a63c86f4e6f906988791f3c5258'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.519120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff89a63c86f4e6f906988791f3c5258'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.528039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f96be93b-ba20-4c4a-9d09-75fb02f1f9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff89a63c86f4e6f906988791f3c5258'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.534286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1140/2294] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:35 2022] DELETE /identity/v3/users/aff89a63c86f4e6f906988791f3c5258 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:35.709775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:35.724016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ce74a-a394-4888-9a14-c9e83e69bba6 service placement] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:35.783523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:35.784165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:35.784443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:35.784718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] PATH_INFO: `/v3/projects/34cb634f5c154187a943a454a706f200` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:35.785895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cb634f5c154187a943a454a706f200)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:35.787794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:35.832475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cb634f5c154187a943a454a706f200', 'request_id': 'req-1bffe804-b55d-48cb-9a88-4bebe17d3921'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.841426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cb634f5c154187a943a454a706f200', 'request_id': 'req-1bffe804-b55d-48cb-9a88-4bebe17d3921'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.847772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cb634f5c154187a943a454a706f200', 'request_id': 'req-1bffe804-b55d-48cb-9a88-4bebe17d3921'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.858211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] The token cache is being invalidate because project 34cb634f5c154187a943a454a706f200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:35.858612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.859236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1bffe804-b55d-48cb-9a88-4bebe17d3921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:35.860743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1155/2295] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:35 2022] DELETE /identity/v3/projects/34cb634f5c154187a943a454a706f200 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:35.866153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:35.866537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:35.866815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:35.905751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:35.926469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:35.927561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:35.985434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1141/2296] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:35.994793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:35.995076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:35.995364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.018095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:36.023852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:36.025671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:36.034886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1156/2297] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:36.039231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.048777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe671aac-a9c7-443b-b379-ad72f1168123 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.053259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.053954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.054238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.054510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.055194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:19:36.055493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.057098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.068798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1142/2298] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:19:36.073026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.083258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc1694-7ea7-4fc6-b358-cf0dd8371243 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.087893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.088501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.088778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.089047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.089753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.091690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.117857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1157/2299] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:36 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:19:36.122402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.132076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b91ce-b192-46dc-8ef3-ef8cdca12cd5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.196680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.197217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.197500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.197777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.198479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.200959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.221855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1143/2300] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:36 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:36.225957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.236143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a32a-2044-4c79-98ef-d27e93b251a8 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.240717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.241317 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.241596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.241971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.242675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.244256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.255845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1158/2301] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:36.259265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.269722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de9ad-4335-487a-a368-e45c5b5b528e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.274488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.275129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.275417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.275694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.276394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.278571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7df73-a706-48bf-861f-dbb1f90b9e40 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.288427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1144/2302] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:36.292346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.302622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43594-273e-4cfa-ab39-980ea09e1bdc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.307171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.307840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.308095 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.308363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] PATH_INFO: `/v3/projects/481ef05fe4444c3caf20f39264088add/users/dfc92f9a448e42d986327578895d6aab/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.322334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] RBAC: Authorizing `identity:create_grant(project_id=481ef05fe4444c3caf20f39264088add, user_id=dfc92f9a448e42d986327578895d6aab, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.325039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.337701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1159/2303] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:36 2022] PUT /identity/v3/projects/481ef05fe4444c3caf20f39264088add/users/dfc92f9a448e42d986327578895d6aab/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:36.357551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.358119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.358510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.387457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] MFA Rules not processed for user `dfc92f9a448e42d986327578895d6aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:36.388678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:36.443444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:36.459298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1145/2304] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:36.467453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.469284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:36.574896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9486d7-80b0-4ec3-9196-53e99d70411a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.580743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.581365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.581649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.581952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.589626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.591259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35558126-76af-4a6a-8505-f6143db8ad59 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.599949 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1160/2305] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:36.804743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:36.865503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb26fc-f657-4290-974f-fab19e62a587 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:36.870050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:36.870744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.871033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.871309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] PATH_INFO: `/v3/users/dfc92f9a448e42d986327578895d6aab` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.872466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc92f9a448e42d986327578895d6aab)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:36.874467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:36.927346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc92f9a448e42d986327578895d6aab'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:36.927346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc92f9a448e42d986327578895d6aab'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:36.937430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc92f9a448e42d986327578895d6aab'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:36.944386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc92f9a448e42d986327578895d6aab'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:36.951330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc92f9a448e42d986327578895d6aab'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:36.958045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1146/2306] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:36 2022] DELETE /identity/v3/users/dfc92f9a448e42d986327578895d6aab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:36.962529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:36.962818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:36.963100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:36.983137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:36.984347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:37.042514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1161/2307] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:37.052676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.054080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:37.120747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002bf94-50b3-4e53-83fe-915bdf9b15fa admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.124506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.125128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.125412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.125685 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.134642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.136074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33469b9c-304a-4bf0-a974-279a7fe3ce94 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.145267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1147/2308] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:37.411933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.413500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:37.420592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:37.427265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe02a-080e-436e-afae-377f9680dc16 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.432733 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.433625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.433956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.434234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] PATH_INFO: `/v3/projects/481ef05fe4444c3caf20f39264088add` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.435404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481ef05fe4444c3caf20f39264088add)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.437328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.485564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481ef05fe4444c3caf20f39264088add', 'request_id': 'req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:37.492422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481ef05fe4444c3caf20f39264088add', 'request_id': 'req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:37.498686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481ef05fe4444c3caf20f39264088add', 'request_id': 'req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:37.512935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] The token cache is being invalidate because project 481ef05fe4444c3caf20f39264088add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:37.513220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:37.513878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8428ee14-74ff-4fb8-91e9-63d6b97fdce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:37.515388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1162/2309] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:37 2022] DELETE /identity/v3/projects/481ef05fe4444c3caf20f39264088add => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:37.520676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.521164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.521276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.530295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:37.541917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:37.543553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:37.600360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1148/2310] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:19:37.608449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.608753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.609025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.628856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:37.630080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:37.638628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1163/2311] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:37.643001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.652586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a0ed-63a8-4290-b1d8-e8abfa27edf1 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.657132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.657736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.658044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.658318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.659008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:19:37.659422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.661017 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.672091 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1149/2312] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:19:37.676387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.686659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ee5ca-aa63-4b70-a17f-c9bd8c005550 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.693239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.693959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.694249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.694527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.695205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.697121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.724801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1164/2313] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:37 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:37.729363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.739712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097343ab-a42b-4978-81aa-25a68136087b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.794634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.795237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.795514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.795784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.796441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.798366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.818784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1150/2314] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:37 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:37.822985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.833866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e334-1fd5-4df7-8ccb-70890a9ac120 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.838364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.838962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.839237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.839630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.840322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.841930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.851443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1165/2315] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:19:37.855155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.865064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2e9e-4058-4693-bedf-f706f8876f72 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.869720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.870350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.871146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.871462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.872154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.873729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb01fc-a234-4fbd-bf8c-7bef2ba1f5f8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.884339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1151/2316] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:37.888945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:37.898546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969477e-796c-4a12-8870-7f3e270de92f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:37.903066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:37.903713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.903991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.904264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] PATH_INFO: `/v3/projects/f7d90e9b611d48bcb2068af14ccc1ac6/users/3e4ea0dae972460c9608e8ee548be64a/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.920441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d90e9b611d48bcb2068af14ccc1ac6, user_id=3e4ea0dae972460c9608e8ee548be64a, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:37.923248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:37.937719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1166/2317] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:37 2022] PUT /identity/v3/projects/f7d90e9b611d48bcb2068af14ccc1ac6/users/3e4ea0dae972460c9608e8ee548be64a/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:37.948975 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:37.949261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:37.949529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:37.976786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] MFA Rules not processed for user `3e4ea0dae972460c9608e8ee548be64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:37.978040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:38.036436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:38.040313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1152/2318] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:38.045187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.105817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453490e-b0d9-44df-b907-88eb19515dbf admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.110559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.111177 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.111460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.111736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.112398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.114364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.140072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1167/2319] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:38 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:38.144100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.154760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e89cf-de28-4928-9ace-f27420a2b922 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.210097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.210707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.210984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.211251 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.211905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.213802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.235599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1153/2320] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:38 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:38.239582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.250441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823847bd-ec4b-4b51-8b95-822dd3ffcf25 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.255061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.255678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.255962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.256239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.256938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.258589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.259831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:38.273798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1168/2321] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:38.275211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.285986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662bd0-8c7a-4423-8021-803dc30a0a19 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.290775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.291370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.291648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.291920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] PATH_INFO: `/v3/projects/b7722499c1ab431cb855adf754862e3b/users/03bf3cb7642e4f9b97df197e485663a5/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.306698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7722499c1ab431cb855adf754862e3b, user_id=03bf3cb7642e4f9b97df197e485663a5, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.309344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.322002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1154/2322] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:38 2022] PUT /identity/v3/projects/b7722499c1ab431cb855adf754862e3b/users/03bf3cb7642e4f9b97df197e485663a5/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:38.325987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.336332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8715-acdc-40ff-b855-ddf3bb4ef6c9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.391359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.392262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.392542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.392809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.393490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.395398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515da35a-0521-460c-81ce-846fb4ed622f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.405384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1169/2323] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:38.409819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.420478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623b83e-995a-46de-b9f4-c76c32c619ba admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.425001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.425646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.425952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.426225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] PATH_INFO: `/v3/projects/b7722499c1ab431cb855adf754862e3b/users/03bf3cb7642e4f9b97df197e485663a5/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.428119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7722499c1ab431cb855adf754862e3b, user_id=03bf3cb7642e4f9b97df197e485663a5, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.430833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.440640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1155/2324] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:38 2022] PUT /identity/v3/projects/b7722499c1ab431cb855adf754862e3b/users/03bf3cb7642e4f9b97df197e485663a5/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:38.450596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.450884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.451185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.479542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] MFA Rules not processed for user `03bf3cb7642e4f9b97df197e485663a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:38.481382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:38.547946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1170/2325] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:38.558884 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.560760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:38.639821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fe80-0eb2-4e6a-8cc8-8dc4b733f749 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.643613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.644239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.644523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.644798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.654444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.655892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231145aa-8c63-4762-bea4-f899276d635a service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.665334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1156/2326] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:38 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:38.815214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:38.880651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c80b75-aa8b-4d66-b5e6-7defb6b3be46 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:38.887105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:38.887759 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.888039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.888313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] PATH_INFO: `/v3/users/3e4ea0dae972460c9608e8ee548be64a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.889519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4ea0dae972460c9608e8ee548be64a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:38.891493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:38.912867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:38.938940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4ea0dae972460c9608e8ee548be64a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:38.951736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4ea0dae972460c9608e8ee548be64a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:38.959043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4ea0dae972460c9608e8ee548be64a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:38.959752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4ea0dae972460c9608e8ee548be64a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:38.960378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4ea0dae972460c9608e8ee548be64a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:38.974139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1171/2327] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:38 2022] DELETE /identity/v3/users/3e4ea0dae972460c9608e8ee548be64a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:38.978826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:38.979213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:38.979495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:38.998539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:38.999727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:39.029095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:39.058710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1157/2328] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:39.068125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:39.069509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:39.125810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffb520-d02b-41b2-bc02-932bdf8b9846 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:39.129489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:39.130144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:39.130424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:39.130699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:39.140086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:39.141506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:39.150877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1172/2329] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:39.412005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:39.413446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:39.425952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea09437-8075-464c-bc85-0d97bf3d9539 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:39.430494 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:39.431134 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:39.431412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:39.431687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] PATH_INFO: `/v3/projects/f7d90e9b611d48bcb2068af14ccc1ac6` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:39.432916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d90e9b611d48bcb2068af14ccc1ac6)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:39.435570 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:39.479857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d90e9b611d48bcb2068af14ccc1ac6', 'request_id': 'req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.487167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d90e9b611d48bcb2068af14ccc1ac6', 'request_id': 'req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.493826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d90e9b611d48bcb2068af14ccc1ac6', 'request_id': 'req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.504543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] The token cache is being invalidate because project f7d90e9b611d48bcb2068af14ccc1ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:39.504830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.505452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.506964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1158/2330] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:39 2022] DELETE /identity/v3/projects/f7d90e9b611d48bcb2068af14ccc1ac6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:39.511069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:39.520613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5afff9-4263-4f55-a3b5-8ca600ff8133 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:39.575764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:39.576426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:39.576672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:39.576948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] PATH_INFO: `/v3/users/03bf3cb7642e4f9b97df197e485663a5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:39.578130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bf3cb7642e4f9b97df197e485663a5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:39.580072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:39.626144 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:39.632275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bf3cb7642e4f9b97df197e485663a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.644851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bf3cb7642e4f9b97df197e485663a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.651706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bf3cb7642e4f9b97df197e485663a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.652386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bf3cb7642e4f9b97df197e485663a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.652995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-eece0233-6994-4a47-94f3-b9fdffe261d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bf3cb7642e4f9b97df197e485663a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:39.665267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1173/2331] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:39 2022] DELETE /identity/v3/users/03bf3cb7642e4f9b97df197e485663a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:39.939414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:39.940993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:39.956712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27de59-80e7-438a-8ffe-a4990e1a5d9d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.015102 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.016001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.016309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.016586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] PATH_INFO: `/v3/projects/b7722499c1ab431cb855adf754862e3b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.018161 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7722499c1ab431cb855adf754862e3b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.020709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.068116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7722499c1ab431cb855adf754862e3b', 'request_id': 'req-680842fb-10f1-424d-9fb8-88d552567ee8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:40.075666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7722499c1ab431cb855adf754862e3b', 'request_id': 'req-680842fb-10f1-424d-9fb8-88d552567ee8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:40.082395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7722499c1ab431cb855adf754862e3b', 'request_id': 'req-680842fb-10f1-424d-9fb8-88d552567ee8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:40.083584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:40.094978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] The token cache is being invalidate because project b7722499c1ab431cb855adf754862e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:40.095262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:40.095886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-680842fb-10f1-424d-9fb8-88d552567ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:40.097379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1159/2332] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:39 2022] DELETE /identity/v3/projects/b7722499c1ab431cb855adf754862e3b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:40.102398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.102686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.102961 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.122736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:40.123968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:40.182856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1174/2333] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:40.190444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.190732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.191002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.210505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:40.211742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:40.220108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1160/2334] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:40.224195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.233861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09222b1-88d1-4198-8365-db4359f9fe4a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.238499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.239086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.239372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.239664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.240339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:19:40.240719 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.242668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.254851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1175/2335] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:19:40.259061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.269271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325094d-b506-46b0-834a-25274d1f3ce0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.273881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.274490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.274772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.275046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.275783 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.278293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.304740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1161/2336] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:40.309307 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.319279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c27630-a028-4e40-9500-bc47605d618f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.374969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.375658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.375938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.376208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.376870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.378826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.399786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1176/2337] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:40.404831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.415184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977339-9f62-46b0-80bc-d3b25b275ecd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.419756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.420358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.420642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.420913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.421599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.423220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.432825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1162/2338] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:19:40.436620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.446349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f55e-467e-4e84-878e-39c407702b46 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.451360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.451959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.452237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.452512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.453197 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.454800 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3085bcd-5e1b-47c7-a171-db84d360cc10 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.465741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1177/2339] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:40.469773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.479810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97705918-29c8-47e2-9bd7-9d71e12f27b0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.484430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.485077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.485359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.485634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] PATH_INFO: `/v3/projects/f1de57a854924512b1e3ca383cef812c/users/7fcff183b36043eb85477d4b4b4c6669/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.499813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1de57a854924512b1e3ca383cef812c, user_id=7fcff183b36043eb85477d4b4b4c6669, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.502486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.516131 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1163/2340] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:40 2022] PUT /identity/v3/projects/f1de57a854924512b1e3ca383cef812c/users/7fcff183b36043eb85477d4b4b4c6669/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:40.528188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.528453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.528723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.555851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] MFA Rules not processed for user `7fcff183b36043eb85477d4b4b4c6669`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:40.557057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:40.569354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:40.628267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1178/2341] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:40.632924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.694460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499231f-5679-44d3-9eab-f27802e5e6a3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.699763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.700389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.700670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.700947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.701624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.703582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.730362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1164/2342] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:19:40.734134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.744859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c6fa2-2446-4a4f-b4bb-41495139bf59 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.802541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.803170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.803457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.803786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.804460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.806420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.828123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1179/2343] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:40.832485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.843007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d0e3-ecfd-4db7-8ae1-a0bcaf0b7e1b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.847625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.848239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.848520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.848794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.849502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.851111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.861601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1165/2344] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:40.865596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.876396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfaaa3-6449-4569-a014-e49362e81d46 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.881037 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.881680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.882007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.882287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.882989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.884618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf3fb8-01a9-47dc-8370-fc4a50040ebc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.895052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1180/2345] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:40.899030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:40.909390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57823f-4a6a-4193-a130-f6fb3f89ddca admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:40.914285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:40.914945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.915225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.915500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] PATH_INFO: `/v3/projects/b9c5fd95ce69459b91f7359a4665289f/users/f5617a84f34c4743a8d718b6089d99f0/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.931997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c5fd95ce69459b91f7359a4665289f, user_id=f5617a84f34c4743a8d718b6089d99f0, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:40.935320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:40.948496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1166/2346] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:40 2022] PUT /identity/v3/projects/b9c5fd95ce69459b91f7359a4665289f/users/f5617a84f34c4743a8d718b6089d99f0/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:40.962936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:40.963239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:40.963516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:40.992022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] MFA Rules not processed for user `f5617a84f34c4743a8d718b6089d99f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:40.993221 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:41.058108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1181/2347] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:41.063462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.102633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:41.129812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551943cd-c203-4dca-ab54-3f72cf6036aa admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.134594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.135234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.135518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.135792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.136512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.138459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.167406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1167/2348] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:41 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:41.172191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.183899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab097631-9f26-4cd8-8197-d4613719218c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.201616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:41.245564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.246924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.247275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.247644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.248597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.250636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.273878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1182/2349] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:41 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:19:41.276295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.287562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5dd77-53dd-429c-bf9e-2139d190a4e1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.293220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.293869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.294158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.294434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.295143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.296754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.308455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1168/2350] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:41.312788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.322805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864adbb-efc3-4192-85e0-2d95c4de364a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.327370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.328033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.328321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.328598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] PATH_INFO: `/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca/users/f0f7458e66384dd996570577b3f40dec/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.343684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa26c316a2fb4d07aef8c8ede24f61ca, user_id=f0f7458e66384dd996570577b3f40dec, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.346396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.363340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1183/2351] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:41 2022] PUT /identity/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca/users/f0f7458e66384dd996570577b3f40dec/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:41.367521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.377460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cbc23-3a2f-49bd-a20c-550eccf3a85c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.431023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.431628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.431910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.432184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.432868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.434522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cae85-7795-4aed-99c5-7cd00a33ce82 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.443827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1169/2352] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:41.447853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.458313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786d19e-70c2-4d64-bafb-88036a69a1d9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.462885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.463545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.463826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.464102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] PATH_INFO: `/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca/users/f0f7458e66384dd996570577b3f40dec/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.466798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa26c316a2fb4d07aef8c8ede24f61ca, user_id=f0f7458e66384dd996570577b3f40dec, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.469426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.479510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1184/2353] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:41 2022] PUT /identity/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca/users/f0f7458e66384dd996570577b3f40dec/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:41.490820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.491179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.491461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.518104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] MFA Rules not processed for user `f0f7458e66384dd996570577b3f40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:41.519300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:41.584807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1170/2354] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:41.596004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:41.597761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:41.654985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c8202-77e7-4e9a-9a61-8d298fad44a9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:41.658865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:41.659478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:41.659758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:41.660064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:41.669607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:41.671067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:41.680903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1185/2355] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:42.178806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:42.180244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:42.192319 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8927b04-a2dc-4802-ae48-a04bd965c6fd None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:42.195886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:42.196491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:42.196774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:42.197050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:42.225789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:42.254785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:42.256223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:42.265583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1171/2356] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:46.271596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:46.280854 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ab1ad-6af1-4da1-90aa-1e4d32f0ab21 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:46.284653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:46.285262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:46.285538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:46.285810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:46.341671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:46.343134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62c4c7-88ee-425f-9b72-17a06a881eee service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:46.351904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1186/2357] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:48.427501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:48.487271 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6751b9-3edb-4c09-8b86-c8d3769661cd service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:48.491809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:48.492375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:48.492655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:48.492929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] PATH_INFO: `/v3/users/7fcff183b36043eb85477d4b4b4c6669` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:48.494116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcff183b36043eb85477d4b4b4c6669)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:48.496023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:48.540639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcff183b36043eb85477d4b4b4c6669'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:48.541277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcff183b36043eb85477d4b4b4c6669'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:48.553287 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcff183b36043eb85477d4b4b4c6669'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:48.559980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcff183b36043eb85477d4b4b4c6669'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:48.567452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcff183b36043eb85477d4b4b4c6669'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:48.573387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1172/2358] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:48 2022] DELETE /identity/v3/users/7fcff183b36043eb85477d4b4b4c6669 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:48.578092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:48.578594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:48.578694 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:48.597279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:48.598507 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:48.607484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:48.657370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1187/2359] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:19:48.667328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:48.668800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:48.722611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:48.726274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac25bb-bc1d-425c-b9d0-17439dd336f9 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:48.731066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:48.731747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:48.732205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:48.732641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:48.742310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:48.743677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:48.752956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1173/2360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:49.018045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:49.019461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:49.032816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2b526-6812-4546-ba82-79751e558cb5 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:49.037485 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:49.038586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:49.038586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:49.038829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] PATH_INFO: `/v3/projects/f1de57a854924512b1e3ca383cef812c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:49.040034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1de57a854924512b1e3ca383cef812c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:49.041896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:49.089154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1de57a854924512b1e3ca383cef812c', 'request_id': 'req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.096976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1de57a854924512b1e3ca383cef812c', 'request_id': 'req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.103391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1de57a854924512b1e3ca383cef812c', 'request_id': 'req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.113817 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] The token cache is being invalidate because project f1de57a854924512b1e3ca383cef812c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:49.114198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.114831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.116336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1188/2361] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:49 2022] DELETE /identity/v3/projects/f1de57a854924512b1e3ca383cef812c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:49.120395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:49.130069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8ab6-4984-43a1-b433-f2c0956f24af service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:49.184350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:49.184978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:49.185259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:49.185529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] PATH_INFO: `/v3/users/f5617a84f34c4743a8d718b6089d99f0` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:49.186724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5617a84f34c4743a8d718b6089d99f0)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:49.188622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:49.233115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5617a84f34c4743a8d718b6089d99f0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.233755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5617a84f34c4743a8d718b6089d99f0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.245253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5617a84f34c4743a8d718b6089d99f0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.252340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5617a84f34c4743a8d718b6089d99f0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.259096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5617a84f34c4743a8d718b6089d99f0'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.265455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1174/2362] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:49 2022] DELETE /identity/v3/users/f5617a84f34c4743a8d718b6089d99f0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:49.527741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:49.529325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:49.542241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca292b-9cc8-49a9-824d-f8ba9c428dae admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:49.598774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:49.599434 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:49.599682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:49.599953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] PATH_INFO: `/v3/projects/b9c5fd95ce69459b91f7359a4665289f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:49.601108 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c5fd95ce69459b91f7359a4665289f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:49.603035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:49.642556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:49.649059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c5fd95ce69459b91f7359a4665289f', 'request_id': 'req-06f58f60-afc6-47cb-aab3-78741c9185d9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.656337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c5fd95ce69459b91f7359a4665289f', 'request_id': 'req-06f58f60-afc6-47cb-aab3-78741c9185d9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.662336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c5fd95ce69459b91f7359a4665289f', 'request_id': 'req-06f58f60-afc6-47cb-aab3-78741c9185d9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.672340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] The token cache is being invalidate because project b9c5fd95ce69459b91f7359a4665289f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:49.672626 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.673241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.674781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1189/2363] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:49 2022] DELETE /identity/v3/projects/b9c5fd95ce69459b91f7359a4665289f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:49.679016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:49.680567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:49.693827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bdb-6128-49f3-ac4e-7847a91ff8ff admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:49.749369 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:49.750044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:49.750324 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:49.750597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] PATH_INFO: `/v3/users/f0f7458e66384dd996570577b3f40dec` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:49.751738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f7458e66384dd996570577b3f40dec)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:49.753653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:49.767138 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:49.801924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f7458e66384dd996570577b3f40dec'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.802574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f7458e66384dd996570577b3f40dec'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.814597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f7458e66384dd996570577b3f40dec'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.822440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f7458e66384dd996570577b3f40dec'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.830068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-437167f3-7ef8-452c-84ba-88181a7e52b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f7458e66384dd996570577b3f40dec'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:49.836158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1175/2364] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:49 2022] DELETE /identity/v3/users/f0f7458e66384dd996570577b3f40dec => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:50.109942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.111479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:50.126510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58f60-afc6-47cb-aab3-78741c9185d9 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.182895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.183546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.183829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.184103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] PATH_INFO: `/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.185258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa26c316a2fb4d07aef8c8ede24f61ca)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.187855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.232668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa26c316a2fb4d07aef8c8ede24f61ca', 'request_id': 'req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:50.240925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa26c316a2fb4d07aef8c8ede24f61ca', 'request_id': 'req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:50.247893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa26c316a2fb4d07aef8c8ede24f61ca', 'request_id': 'req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:50.258023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] The token cache is being invalidate because project fa26c316a2fb4d07aef8c8ede24f61ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:50.258312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:50.258947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f847c4a4-87b0-4dc5-9b4d-71daa05be43c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:50.260560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1190/2365] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:50 2022] DELETE /identity/v3/projects/fa26c316a2fb4d07aef8c8ede24f61ca => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:19:50.266483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.266798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.267079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.286992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:50.288185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:50.346320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1176/2366] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:50.354546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.354828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.355101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.375187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:50.376416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:50.384355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1191/2367] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:50.388474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.398036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998ed3b-b49a-443b-ad70-99da04c97da2 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.402671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.403282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.403563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.403842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.405050 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:19:50.405342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.406997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.418380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1177/2368] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:19:50.423153 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.432114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d229-ed95-40e9-9669-0501c7d5edd0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.436692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.437294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.437580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.437867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.438540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.440427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.466541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1192/2369] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:50 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:19:50.471262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.481305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744c49-266c-4cb1-a8c4-adad252e1ddb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.535437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.536051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.536346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.536613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.537313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.539262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.560356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1178/2370] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:50 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:50.564513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.574805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63916f2c-75c1-41f8-a3eb-d30e0e299925 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.580000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.580868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.581150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.581427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.582263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.583828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.594243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1193/2371] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:50.598110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.607873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1737c1-ef6c-41b3-aab0-278a835b0eb6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.612557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.613160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.613443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.613717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.614463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.616042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec8c5b-dfd4-4967-aed0-c056bb9cfeb3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.617261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:50.628673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1179/2372] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:50.632809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.643081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc636e70-c199-46d4-8106-d8041e817be2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.647697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.648344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.648621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.648894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] PATH_INFO: `/v3/projects/a68575a7c4b649ad926fde3bef01ccea/users/d4d1902a4ac1485481264da6c510882b/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.663634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68575a7c4b649ad926fde3bef01ccea, user_id=d4d1902a4ac1485481264da6c510882b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.666316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.669236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:50.681803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1194/2373] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:50 2022] PUT /identity/v3/projects/a68575a7c4b649ad926fde3bef01ccea/users/d4d1902a4ac1485481264da6c510882b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:50.693158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.693637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.693735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.721612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] MFA Rules not processed for user `d4d1902a4ac1485481264da6c510882b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:50.722858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:50.784898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1180/2374] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:50.796342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.798059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:50.854633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b29e-a019-4da7-b605-fdeafa889004 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.858308 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.858930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.859208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.859479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.867715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.869131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763cd60-20c7-447a-b772-b8b22c7f954d service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:50.877907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1195/2375] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:50.897116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:50.963880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89527-3380-42f8-b2e9-9e1e17caa8b0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:50.969283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:50.969952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:50.970237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:50.970512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] PATH_INFO: `/v3/users/d4d1902a4ac1485481264da6c510882b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:50.971676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d1902a4ac1485481264da6c510882b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:50.973587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.026298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d1902a4ac1485481264da6c510882b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.026977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d1902a4ac1485481264da6c510882b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.039103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d1902a4ac1485481264da6c510882b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.046838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d1902a4ac1485481264da6c510882b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.054601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d1902a4ac1485481264da6c510882b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.061969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1181/2376] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:19:50 2022] DELETE /identity/v3/users/d4d1902a4ac1485481264da6c510882b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:51.066516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.066808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.067083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.089174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:51.090446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:51.153750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1196/2377] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:51.163930 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.165363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:51.181094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:51.227861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24dcd-1e16-4b54-8ac9-fd0594d0d2bb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.231416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.232030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.232309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.232583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.241312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.242867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdcc48-0f8c-46f7-a0ba-f4e1391132da service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.252569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1182/2378] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:19:51.526658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.528086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:51.543582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7c4bb-a4d7-4d6b-b223-272384a4bd54 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.550723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.551334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.551711 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.551878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] PATH_INFO: `/v3/projects/a68575a7c4b649ad926fde3bef01ccea` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.553175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68575a7c4b649ad926fde3bef01ccea)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.555174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.600810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68575a7c4b649ad926fde3bef01ccea', 'request_id': 'req-d5b43b76-6314-4fd6-86ed-baac197f5002'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.608412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68575a7c4b649ad926fde3bef01ccea', 'request_id': 'req-d5b43b76-6314-4fd6-86ed-baac197f5002'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.614803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68575a7c4b649ad926fde3bef01ccea', 'request_id': 'req-d5b43b76-6314-4fd6-86ed-baac197f5002'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.625526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] The token cache is being invalidate because project a68575a7c4b649ad926fde3bef01ccea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:19:51.625808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.626478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d5b43b76-6314-4fd6-86ed-baac197f5002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:19:51.627991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1197/2379] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:19:51 2022] DELETE /identity/v3/projects/a68575a7c4b649ad926fde3bef01ccea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:51.634232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.634518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.634793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.653533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:51.654806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:51.714904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1183/2380] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:51.724508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.724939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.725036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.744943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:51.746173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:51.753948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1198/2381] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:51.758166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.767772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bd3f3-174a-4f2c-96e4-fbdcc6a491f6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.772367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.773002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.773276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.773543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.774275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:19:51.774575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.776152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.787831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1184/2382] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:19:51.791895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.802348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9937671-3a3b-47e7-a111-19ac29c458c7 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.807134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.807783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.808161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.808437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.809102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.811060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.825715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:51.838746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1199/2383] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:19:51 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:19:51.843367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.853754 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5967b-dd5f-486b-bb23-407ac450fd2d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.911339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.911870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.912143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.912415 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.913072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.915007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.938571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1185/2384] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:19:51 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:19:51.939780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.950141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af356f3-acd5-4dc8-9a6f-c8e60befd96f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.954776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.955390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.955937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.956215 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.956944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.958587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:51.968878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1200/2385] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:51.972672 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:51.982657 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba626f45-be2b-40a3-be62-09d3e232d8cb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:51.987943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:51.988547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:51.988826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:51.989101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:51.989787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:51.991416 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352515e-cd7e-4577-b96c-3703b693cf0e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:52.001191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1186/2386] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:19:52.005240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:52.015084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c3614-16cf-4f7d-a1b1-e0deddce45e3 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:52.019712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:52.020369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:52.020655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:52.020936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] PATH_INFO: `/v3/projects/be97518582974331b49a7af74ed58801/users/86e9c107062f44e396f1f6d52203fdf1/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:52.035638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be97518582974331b49a7af74ed58801, user_id=86e9c107062f44e396f1f6d52203fdf1, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:52.038340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:52.051887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1201/2387] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:19:52 2022] PUT /identity/v3/projects/be97518582974331b49a7af74ed58801/users/86e9c107062f44e396f1f6d52203fdf1/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:19:52.116564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:52.116886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:52.117162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:52.142180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:19:52.145202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] MFA Rules not processed for user `86e9c107062f44e396f1f6d52203fdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:19:52.146978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:52.209859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1187/2388] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:19:52.221656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:19:52.223644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:19:52.281566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519764-048b-41df-8831-0b2d5487f81e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:19:52.285495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:19:52.286137 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:19:52.286417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:19:52.286689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:19:52.295006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:19:52.296482 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42cd90-7951-4eaa-8f3b-57a8d205827b service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:19:52.305422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1202/2389] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:20:01.386326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:01.448825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81a091-9b75-41ab-a8bd-f72a0876306f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:01.453243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:01.453896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:01.454194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:01.454448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] PATH_INFO: `/v3/users/86e9c107062f44e396f1f6d52203fdf1` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:01.455582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e9c107062f44e396f1f6d52203fdf1)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:01.457736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:01.502343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e9c107062f44e396f1f6d52203fdf1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:01.502969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e9c107062f44e396f1f6d52203fdf1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:01.515622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e9c107062f44e396f1f6d52203fdf1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:01.522709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e9c107062f44e396f1f6d52203fdf1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:01.530694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e9c107062f44e396f1f6d52203fdf1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:01.536824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1188/2390] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:20:01 2022] DELETE /identity/v3/users/86e9c107062f44e396f1f6d52203fdf1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:01.541349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:01.541832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:01.541969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:01.560994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:01.562213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:01.619457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1203/2391] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:01.628931 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:01.630330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:01.655874 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:01.689189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22ea3e-62ee-4ad9-8677-57a85ceda08b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:01.692848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:01.693466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:01.693748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:01.694053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:01.702907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:01.704327 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45025b-9836-46cd-a0b7-d0ceac591003 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:01.713834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1189/2392] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:20:01.985669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:01.987234 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:01.998950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b68de5-a57c-4c10-a047-ef8e678e0502 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.003479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.004109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.004386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.004657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] PATH_INFO: `/v3/projects/be97518582974331b49a7af74ed58801` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.005799 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be97518582974331b49a7af74ed58801)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.008018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.056729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be97518582974331b49a7af74ed58801', 'request_id': 'req-40d6a06b-19fb-401d-88a9-90bb5d779905'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:02.064106 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be97518582974331b49a7af74ed58801', 'request_id': 'req-40d6a06b-19fb-401d-88a9-90bb5d779905'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:02.065279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:02.071764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be97518582974331b49a7af74ed58801', 'request_id': 'req-40d6a06b-19fb-401d-88a9-90bb5d779905'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:02.083382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] The token cache is being invalidate because project be97518582974331b49a7af74ed58801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:20:02.083673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:02.084335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-40d6a06b-19fb-401d-88a9-90bb5d779905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:20:02.085923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1204/2393] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:20:01 2022] DELETE /identity/v3/projects/be97518582974331b49a7af74ed58801 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:02.091288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.091578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.091857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.111389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:02.112575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:02.169648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1190/2394] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:20:02.177342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.177635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.177939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.197201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:02.198443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:02.207136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1205/2395] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:02.211428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.221707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a048a3-468b-4afe-b938-ddfb1f664e60 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.226260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.226865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.227143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.227419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.228108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:20:02.228412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.230032 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.241219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1191/2396] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:20:02.245330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.255159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d72d99-c4ca-49b6-866f-c471cbd25f10 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.259814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.260422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.260703 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.260980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.261650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.263600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.291743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1206/2397] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:20:02 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:20:02.296528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.306437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1e833-a8f2-4467-9d7d-fe9f86bf58d0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.360677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.361284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.361566 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.361865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.362527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.364439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.386506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1192/2398] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:20:02 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:20:02.390804 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.401394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cfaf2-03d8-4366-8c86-41290e459742 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.406559 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.407169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.407449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.407723 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.408425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.410041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.419995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1207/2399] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:20:02.424383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.435915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156bb2-28ae-40ca-a25c-5fd5eb9a748b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.440623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.441224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.441502 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.441776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.442499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.444079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563ca9-253f-430e-a0db-b42f94d18bc8 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.454417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1193/2400] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:20:02.458634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.468735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43249-d46d-4214-97ec-a468c002922f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.473220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.473885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.474167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.474439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] PATH_INFO: `/v3/projects/53fd239676ec42e0a9f145e924fc249a/users/bcc41a77cdfb426caa5c86ecab2298e8/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.489983 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53fd239676ec42e0a9f145e924fc249a, user_id=bcc41a77cdfb426caa5c86ecab2298e8, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.492607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.504824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1208/2401] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:20:02 2022] PUT /identity/v3/projects/53fd239676ec42e0a9f145e924fc249a/users/bcc41a77cdfb426caa5c86ecab2298e8/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:20:02.510010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f35c-1b8c-47f8-b085-5f88522983d0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.510301 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f35c-1b8c-47f8-b085-5f88522983d0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.510578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f35c-1b8c-47f8-b085-5f88522983d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.528950 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-3a36f35c-1b8c-47f8-b085-5f88522983d0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:02.530695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3a36f35c-1b8c-47f8-b085-5f88522983d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:02.555873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:02.590328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1194/2402] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:02.600629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:02.602421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:02.658591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55752019-019c-4106-86c5-524e10251cb2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:02.662277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:02.662897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:02.663181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:02.663459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:02.672442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:02.673812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:02.684514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1209/2403] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:20:06.580747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:06.581515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:06.581612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:06.611961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] MFA Rules not processed for user `bcc41a77cdfb426caa5c86ecab2298e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:06.613431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:06.677968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1195/2404] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:06.682744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:06.693138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd714d-d6f6-42c5-b935-238bc435df6c service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:06.705662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:06.713366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:06.714128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:06.714128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:06.714743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:06.716749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:06.749550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1210/2405] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:20:06 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:06.750829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:06.762308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82754d-2d34-447a-b301-0e20aedc8a3d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:06.823170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:06.823792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:06.824069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:06.824341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:06.825031 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:06.827703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:06.850880 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1196/2406] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:20:06 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:20:06.855007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:06.866023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a741-56ae-4d13-8e3f-7dc750a09b2a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:06.875950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:06.876588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:06.876869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:06.877142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:06.886634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:06.888315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:06.906819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1211/2407] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:20:06.911059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:06.921154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9e61-2bd9-44cd-adee-7154d6752a85 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:06.936228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:06.936989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:06.937270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:06.937547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] PATH_INFO: `/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe/users/363f198e18094e0e84d60f63d3443622/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:06.955156 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e81acb21c8b4a439a0ee4ea3ce5aebe, user_id=363f198e18094e0e84d60f63d3443622, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:06.958425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:06.974844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1197/2408] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:20:06 2022] PUT /identity/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe/users/363f198e18094e0e84d60f63d3443622/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:20:06.978976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:06.989486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb64b7e-5ace-478f-94ff-b180d3963a3e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:07.043969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:07.056258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:07.056898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:07.057208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:07.057488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:07.058229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:07.059834 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d42cc-e7fc-4f37-945a-5fe5df971faa admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:07.079180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1212/2409] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:20:07.083629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:07.096537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1160-d40c-4153-857d-466a215f45ee admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:07.102849 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:07.103583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:07.103865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:07.104137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] PATH_INFO: `/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe/users/363f198e18094e0e84d60f63d3443622/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:07.107346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e81acb21c8b4a439a0ee4ea3ce5aebe, user_id=363f198e18094e0e84d60f63d3443622, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:07.110084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:07.121019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1198/2410] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:20:07 2022] PUT /identity/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe/users/363f198e18094e0e84d60f63d3443622/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:20:11.371015 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34433f5-6cc9-4ecf-b823-fff358498563 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:11.371936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34433f5-6cc9-4ecf-b823-fff358498563 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:11.372030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34433f5-6cc9-4ecf-b823-fff358498563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:11.406904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c34433f5-6cc9-4ecf-b823-fff358498563 None None] MFA Rules not processed for user `363f198e18094e0e84d60f63d3443622`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:11.408499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c34433f5-6cc9-4ecf-b823-fff358498563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:11.494026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1213/2411] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:20:11.512313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:11.515493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:11.530489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:11.600721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b9bb-9ce3-459a-bd9c-c70ee28c73b1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:11.606402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:11.607089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:11.607276 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:11.607581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:11.671524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:11.673345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cad79-f088-4f45-b89e-c0c823a5d063 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:11.684426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1199/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:20:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:20:12.818849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a895-9e62-4104-8d48-22625837ea05 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:12.819389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a895-9e62-4104-8d48-22625837ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:12.819389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a895-9e62-4104-8d48-22625837ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:12.839486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8162a895-9e62-4104-8d48-22625837ea05 None None] MFA Rules not processed for user `bcc41a77cdfb426caa5c86ecab2298e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:12.840690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8162a895-9e62-4104-8d48-22625837ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:12.855909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1214/2413] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:21.678871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=50 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:22.837345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=43 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:43.103670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:43.104204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:43.104308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:43.124413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] MFA Rules not processed for user `e0c1ed47784c46baa7ddc4abdfc54bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:43.125647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:43.182041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1200/2414] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:53.175267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=25 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:20:57.965023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:57.965521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:57.965625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:57.998686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:20:57.999892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:58.057337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1215/2415] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:20:58.103644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:20:58.105204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:20:58.180117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae90fee-88c3-4b71-a6f8-f4ac18826ed6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:20:58.185119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:20:58.185748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:20:58.186061 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:20:58.186339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:20:58.196353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:20:58.198209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cc538-dc5a-4f48-94a6-60053b4571f0 service swift] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:20:58.207920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1201/2416] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:20:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:21:00.952512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:00.954731 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:01.051717 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928199d4-27a2-40a2-9e00-7cd01244ba22 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:01.056581 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:01.057300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:01.057653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:01.058017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:01.072230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:01.078576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:01.096866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1216/2417] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:21:08.203736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=32 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:11.087539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=55 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:16.394671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:16.395173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:16.395288 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:16.413761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:16.415010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:16.422185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1202/2418] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:21:16.434566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:16.446319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3381-bdf9-4fda-9073-817253727b1a service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:16.451241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:16.451851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:16.452135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:16.452414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:16.460769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:16.462402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:16.470377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1217/2419] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:21:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:21:19.455687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:19.465999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbc668-bd64-419c-a39d-fdb21bb531e5 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:19.470013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:19.470646 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:19.470927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:19.471199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:19.473199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:19.494544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:19.495852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:19.508577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1203/2420] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:20.703562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:20.714168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ce3d4-1e6b-4f78-8cbd-25d4c4d787d2 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:20.717697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:20.718333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:20.718611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:20.718885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:20.720914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:20.739593 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:20.740986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:20.754229 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1218/2421] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:29.011891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.013281 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.027635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ec64e-1163-4566-8963-8fbd2cfa8351 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.033105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.033658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.033976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.034253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.036492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.114179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.115915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffea259-94ee-4356-90d0-1d9374fe3e74 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:29.124752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1204/2422] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:29.847979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:29.849410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.911019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a47ce-d3f4-42bc-adc0-8cf9139173ab service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:29.914758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:29.915374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:29.915650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:29.915922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:29.917643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:29.997149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:29.998613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:30.008156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1219/2423] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:38.780366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.780855 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.780938 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.787405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=47 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:38.835430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:38.836647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:38.853777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1205/2424] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:38.865797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:38.886622 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d002e48-a911-486d-82de-d53de7c70f4e service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:38.891648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:38.892261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:38.892540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:38.892815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:38.901345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:38.903063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:38.911021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1220/2425] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:21:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:21:47.640334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:47.650216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf729-ae54-40e4-9cbf-16b0ed24f081 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:47.653755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:47.654396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:47.654673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:47.654944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:47.656692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:47.718376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:47.719818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:47.729917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1206/2426] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:48.907830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=79 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:50.807907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:50.809468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:50.823954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfc87e-bfd9-42bf-a74a-b49389d739bf service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:50.828492 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:50.829083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:50.829362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:50.829653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] PATH_INFO: `/v3/users/87b166b84ccd4300a1b862d375c137a5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:50.830843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b166b84ccd4300a1b862d375c137a5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:50.833001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:50.885829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b166b84ccd4300a1b862d375c137a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:50.897598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b166b84ccd4300a1b862d375c137a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:50.904287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b166b84ccd4300a1b862d375c137a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:50.904954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b166b84ccd4300a1b862d375c137a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:50.905562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-60829e94-b7b1-460c-9577-2bcc3660c982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b166b84ccd4300a1b862d375c137a5'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:50.917736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1221/2427] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:21:50 2022] DELETE /identity/v3/users/87b166b84ccd4300a1b862d375c137a5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:51.093054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.103611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6298d5e-556a-41e1-a3cb-0053e6b7a920 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.160670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.161358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.162174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.162450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] PATH_INFO: `/v3/projects/b7b5e309462a425a92e550dd2c72ca4d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.163615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b5e309462a425a92e550dd2c72ca4d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.165558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.210906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b5e309462a425a92e550dd2c72ca4d', 'request_id': 'req-2acd10c6-438a-42a6-92ac-7992c816979c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:51.217785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b5e309462a425a92e550dd2c72ca4d', 'request_id': 'req-2acd10c6-438a-42a6-92ac-7992c816979c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:51.219742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.224519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b5e309462a425a92e550dd2c72ca4d', 'request_id': 'req-2acd10c6-438a-42a6-92ac-7992c816979c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:51.235994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] The token cache is being invalidate because project b7b5e309462a425a92e550dd2c72ca4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:21:51.236279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:51.236906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2acd10c6-438a-42a6-92ac-7992c816979c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:51.238429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1207/2428] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:21:51 2022] DELETE /identity/v3/projects/b7b5e309462a425a92e550dd2c72ca4d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:51.243451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.243740 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.244047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.263542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:51.264728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:51.321453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1222/2429] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:21:51.329313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.329605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.329914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.350661 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:51.351863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:51.359841 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1208/2430] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:51.364333 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.374984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead48d-c234-4d61-8cce-16f11949044a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.379587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.380204 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.380484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.380757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.381453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:51.381758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.383382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.395220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1223/2431] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:51.399439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.408644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc5ada-a777-4b07-aa13-6e30b74f0d3c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.413830 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.414470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.414752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.415026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.415691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.417596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.453317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1209/2432] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:21:51 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:51.457345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.468034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe542b0-33d6-4bc0-ba9a-2a27b6bb7043 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.523964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.524588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.524864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.525136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.525793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.527732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.535610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:51.549640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1224/2433] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:21:51 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:51.554291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.564717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2bff8-edfe-4acd-a5b9-570b53ebaf82 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.570233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.570869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.571149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.571425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.572115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.573713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.584168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1210/2434] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:51.588312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.598213 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54acef-e07c-4a3b-b4cc-b2c685c0bb7b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.603284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.603836 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.604112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.604382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.605142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.607003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdc693-e461-4ebf-b1ac-164bcff8ad34 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.618458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1225/2435] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:51.622833 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.633523 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7fe6-d619-42f6-88b9-bea6b85b2664 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.638242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.639887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.640177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.640457 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] PATH_INFO: `/v3/projects/cdede8a0184746b7ad130e3a305e337d/users/96836fdc7c6b4989a362112db17e672f/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.655413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdede8a0184746b7ad130e3a305e337d, user_id=96836fdc7c6b4989a362112db17e672f, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.658097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.671920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1211/2436] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:21:51 2022] PUT /identity/v3/projects/cdede8a0184746b7ad130e3a305e337d/users/96836fdc7c6b4989a362112db17e672f/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:51.683107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.683438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.683714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.713922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] MFA Rules not processed for user `96836fdc7c6b4989a362112db17e672f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:51.715134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:51.778756 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1226/2437] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:51.794149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:51.796009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:51.873109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b45e-508b-40dc-a136-7db7e991f548 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:51.878594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:51.879206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:51.879484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:51.879757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:51.887908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:51.889482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28222fe-7098-47b3-b6ee-4ef6bb4b4984 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:51.898690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1212/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:21:54.563848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.627901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cb46a-a9ee-4f14-928a-a59b3e6703b4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.633105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.633746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.634060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.634337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] PATH_INFO: `/v3/users/96836fdc7c6b4989a362112db17e672f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.635495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96836fdc7c6b4989a362112db17e672f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.637456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.686707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96836fdc7c6b4989a362112db17e672f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:54.699079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96836fdc7c6b4989a362112db17e672f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:54.703381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.706943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96836fdc7c6b4989a362112db17e672f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:54.707749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96836fdc7c6b4989a362112db17e672f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:54.708928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96836fdc7c6b4989a362112db17e672f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:54.721332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1227/2439] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:21:54 2022] DELETE /identity/v3/users/96836fdc7c6b4989a362112db17e672f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:54.725869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.726376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.726481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.745095 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:54.746361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:54.795160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:54.804782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1213/2440] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:54.814460 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:54.815849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:54.873582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64202ae-da27-432b-bfc6-9a4b54ddc614 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:54.877341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:54.878029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:54.878316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:54.878587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:54.890781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:54.892381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af0bb-363d-4a24-8475-4a09b8cd342c service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:54.902499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1228/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:55.165711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.167219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:55.183991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be069fd-f99a-446f-af5d-92945d3500f0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.189617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.190179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.190461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.190813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] PATH_INFO: `/v3/projects/cdede8a0184746b7ad130e3a305e337d` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.192028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdede8a0184746b7ad130e3a305e337d)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.193955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.243073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdede8a0184746b7ad130e3a305e337d', 'request_id': 'req-6b7c7bc3-deec-4487-a571-e224a4905bd1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:55.250176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdede8a0184746b7ad130e3a305e337d', 'request_id': 'req-6b7c7bc3-deec-4487-a571-e224a4905bd1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:55.256191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdede8a0184746b7ad130e3a305e337d', 'request_id': 'req-6b7c7bc3-deec-4487-a571-e224a4905bd1'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:55.266443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] The token cache is being invalidate because project cdede8a0184746b7ad130e3a305e337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:21:55.266731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:55.267367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-6b7c7bc3-deec-4487-a571-e224a4905bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:55.268871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1214/2442] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:21:55 2022] DELETE /identity/v3/projects/cdede8a0184746b7ad130e3a305e337d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:55.274359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.274644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.274921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.294501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:55.295690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:55.355053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1229/2443] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:55.362965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.363255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.363534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.382859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:55.384062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:55.391929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1215/2444] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:55.396049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.406142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aa555-fc0a-4220-9653-9ba21b45e75b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.411491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.412716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.412995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.413377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.414500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:55.414808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.416661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.431385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1230/2445] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:55.435804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.445080 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ae903-70d9-4f78-9db2-988ddc615902 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.449719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.450347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.450623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.450895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.451549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.453433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.481238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1216/2446] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:55.485456 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.495512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3bba4-c541-4e62-b73a-5cf5736d2dd5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.552377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.553009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.553295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.553571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.554297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.556232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.577699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1231/2447] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:55.581871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.592831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4cac9-4d13-4776-a872-3ff486a7f885 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.597374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.598012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.598292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.598560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.599243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.601508 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.611476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1217/2448] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:55.615459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.630410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcf010-eea2-4164-9157-7b8d7e00f912 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.635819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.636439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.636713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.636984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.637677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.639296 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e9c46-fe3e-485f-9328-9951bc98259e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.640515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:55.654778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1232/2449] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:55.655909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.665820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044226a-6b80-44b6-a849-baa69824f999 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.670630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.671291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.671571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.671847 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] PATH_INFO: `/v3/projects/0c111e10932b4fb5a011fa9191180168/users/832e067fad9049aa9d522326bc35ae7f/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.686484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c111e10932b4fb5a011fa9191180168, user_id=832e067fad9049aa9d522326bc35ae7f, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.689506 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.702471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1218/2450] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:21:55 2022] PUT /identity/v3/projects/0c111e10932b4fb5a011fa9191180168/users/832e067fad9049aa9d522326bc35ae7f/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:21:55.715096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.715386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.715665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.742971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] MFA Rules not processed for user `832e067fad9049aa9d522326bc35ae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:55.744199 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:55.807335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1233/2451] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:55.811725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.871818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba145f-7038-467c-b604-4eb2a0485588 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.876504 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.877117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.877400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.877675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.878355 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.880272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:55.906737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1219/2452] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:55.910881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:55.921691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a611a4-86f6-4a3a-aaa8-2d99bf4cd7da None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:55.976196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:55.976806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:55.977085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:55.977356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:55.978040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:55.979947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.000212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1234/2453] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:21:55 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:21:56.004588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.013438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.017698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089596c-8a81-4088-bb1a-5c7a39baeebd admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.024807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.025417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.025698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.026013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.026709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.028452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.039206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1220/2454] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:56.043461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.054032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598302-d3ee-4aae-a27b-e2ed73aa4c79 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.059187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.059890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.060269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.060641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] PATH_INFO: `/v3/projects/01598555d95e48daadf4e2d73e648ea4/users/5faeed03a5c043c6881b0bf3dadea16e/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.076024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01598555d95e48daadf4e2d73e648ea4, user_id=5faeed03a5c043c6881b0bf3dadea16e, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.078701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.091546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1235/2455] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:21:56 2022] PUT /identity/v3/projects/01598555d95e48daadf4e2d73e648ea4/users/5faeed03a5c043c6881b0bf3dadea16e/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:56.095449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.105366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2b677-a52f-4947-822a-2875c60f4f79 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.159030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.159632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.159910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.160182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.160861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.162621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cca86-3e06-4c87-90b8-1f1443091cfb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.171919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1221/2456] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:56.176036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.186718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cb36d-f07a-4603-9823-71a8b6088831 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.191369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.192027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.192304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.192576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] PATH_INFO: `/v3/projects/01598555d95e48daadf4e2d73e648ea4/users/5faeed03a5c043c6881b0bf3dadea16e/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.195087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=01598555d95e48daadf4e2d73e648ea4, user_id=5faeed03a5c043c6881b0bf3dadea16e, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.197696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.207390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1236/2457] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:21:56 2022] PUT /identity/v3/projects/01598555d95e48daadf4e2d73e648ea4/users/5faeed03a5c043c6881b0bf3dadea16e/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:21:56.217599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.217918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.218201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.245649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] MFA Rules not processed for user `5faeed03a5c043c6881b0bf3dadea16e`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:56.246928 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:56.313325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1222/2458] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:56.324336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:56.326070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:56.367327 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:56.384261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9c511-9904-4649-9195-f788eca209ed admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:56.387954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:56.388583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:56.388884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:56.389141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:56.398455 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:56.400118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb86b28-0f6c-4f2e-9151-2560489254fb service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:56.409781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1237/2459] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:56.990998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.051998 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada728cd-08d2-40aa-bd5a-25cced1e8f4c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.056659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.057291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.057577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.057876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] PATH_INFO: `/v3/users/832e067fad9049aa9d522326bc35ae7f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.059040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=832e067fad9049aa9d522326bc35ae7f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.060962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.076090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.105332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832e067fad9049aa9d522326bc35ae7f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.105940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832e067fad9049aa9d522326bc35ae7f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.117481 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832e067fad9049aa9d522326bc35ae7f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.124388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832e067fad9049aa9d522326bc35ae7f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.131898 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832e067fad9049aa9d522326bc35ae7f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.137714 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1223/2460] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:21:56 2022] DELETE /identity/v3/users/832e067fad9049aa9d522326bc35ae7f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:57.142390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.142875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.142974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.161749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:57.163007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:57.220765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1238/2461] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:57.230617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.232010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:57.288568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcbc5-3c90-4027-857b-27c321433785 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.292235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.292850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.293135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.293412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.302280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.303713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.313341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1224/2462] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:21:57.586401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.587922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:57.600151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a07d3-d48e-4360-81c5-3aad1db40e42 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.604727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.605379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.605664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.605970 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] PATH_INFO: `/v3/projects/0c111e10932b4fb5a011fa9191180168` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.607173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c111e10932b4fb5a011fa9191180168)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.609114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.655381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c111e10932b4fb5a011fa9191180168', 'request_id': 'req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.662391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c111e10932b4fb5a011fa9191180168', 'request_id': 'req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.668783 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c111e10932b4fb5a011fa9191180168', 'request_id': 'req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.680004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] The token cache is being invalidate because project 0c111e10932b4fb5a011fa9191180168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:21:57.680287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.680910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.682458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1239/2463] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:21:57 2022] DELETE /identity/v3/projects/0c111e10932b4fb5a011fa9191180168 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:57.686455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:57.698026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ef00-5894-441a-8bef-551df555a1ae service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:57.753630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:57.754291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:57.754571 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:57.754844 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] PATH_INFO: `/v3/users/5faeed03a5c043c6881b0bf3dadea16e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:57.756859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5faeed03a5c043c6881b0bf3dadea16e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:57.758801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:57.797119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:57.804970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5faeed03a5c043c6881b0bf3dadea16e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.806049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5faeed03a5c043c6881b0bf3dadea16e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.821455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5faeed03a5c043c6881b0bf3dadea16e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.829633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5faeed03a5c043c6881b0bf3dadea16e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.836988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c7a5d1d2-6b67-4e02-afb4-667e70867208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5faeed03a5c043c6881b0bf3dadea16e'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:57.843450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1225/2464] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:21:57 2022] DELETE /identity/v3/users/5faeed03a5c043c6881b0bf3dadea16e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:58.117695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.119406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.132863 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fbb91-6860-46eb-9ff8-78eaccbe6ecd admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.193621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.194294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.194484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.194754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] PATH_INFO: `/v3/projects/01598555d95e48daadf4e2d73e648ea4` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.195921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01598555d95e48daadf4e2d73e648ea4)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.197830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.203887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.245976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01598555d95e48daadf4e2d73e648ea4', 'request_id': 'req-abe815af-decc-45cd-bc85-fc1947ffc3d3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:58.253346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01598555d95e48daadf4e2d73e648ea4', 'request_id': 'req-abe815af-decc-45cd-bc85-fc1947ffc3d3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:58.259543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01598555d95e48daadf4e2d73e648ea4', 'request_id': 'req-abe815af-decc-45cd-bc85-fc1947ffc3d3'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:58.269860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] The token cache is being invalidate because project 01598555d95e48daadf4e2d73e648ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:21:58.270160 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:58.270788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-abe815af-decc-45cd-bc85-fc1947ffc3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:21:58.272801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1240/2465] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:21:58 2022] DELETE /identity/v3/projects/01598555d95e48daadf4e2d73e648ea4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:58.277718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.278221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.278317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.297589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:58.298834 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.356499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1226/2466] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:58.364178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.364465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.364746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.384094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:58.385328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.393286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1241/2467] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:58.397968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.407560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b458140-a2a4-453c-86db-ffa4deff2bc0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.412637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.413240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.413515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.413787 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.414617 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:21:58.414923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.416521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.427862 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1227/2468] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:21:58.431978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.441368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288847b0-5154-4624-a69e-d267a5457f40 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.447129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.447754 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.448035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.448311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.448989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.451920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.481180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1242/2469] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:21:58.485284 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.495406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde3690-d9a8-467b-9e77-fd130e0b796b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.550740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.551356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.551640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.551917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.552587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.554545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.576252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1228/2470] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:21:58.580293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.591368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40391bfe-e094-4a1d-bf9f-fae48b622d94 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.596119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.596739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.597018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.597291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.598033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.600267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.610277 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1243/2471] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:58.614103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.624244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaff5d4-c9a1-4196-8de6-5d9afb988726 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.628818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.629419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.629697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.630016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.631246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.632840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea281823-d940-4712-b392-d727fff9d4cd admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.643185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1229/2472] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:21:58.647851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.657797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc4df-90ff-450b-a2bf-7a136fd0cdb5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.662477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.663131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.663410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.663684 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] PATH_INFO: `/v3/projects/1180260701324813ab68960aa4bdbe19/users/4f59ec9365de4de284acad755b7c5f18/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.677773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1180260701324813ab68960aa4bdbe19, user_id=4f59ec9365de4de284acad755b7c5f18, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.680523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.694700 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1244/2473] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:21:58 2022] PUT /identity/v3/projects/1180260701324813ab68960aa4bdbe19/users/4f59ec9365de4de284acad755b7c5f18/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:21:58.705891 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.706314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.706628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.733971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] MFA Rules not processed for user `4f59ec9365de4de284acad755b7c5f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:21:58.735159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.744059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:21:58.800717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1230/2474] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:21:58.811384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:21:58.813063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:21:58.869447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbe11-42ba-41f5-a3e5-3a8c09aa7388 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:21:58.873159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:21:58.873782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:21:58.874099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:21:58.874377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:21:58.883295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:21:58.884808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a2be2-1f11-4770-b2a9-91f28b5c3344 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:21:58.893488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1245/2475] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:21:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:08.319644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.380193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb327ff-cfe4-42df-ad29-a9f7e040c09e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.385370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.386044 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.386329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.386628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] PATH_INFO: `/v3/users/4f59ec9365de4de284acad755b7c5f18` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.387789 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f59ec9365de4de284acad755b7c5f18)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.389710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.435480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f59ec9365de4de284acad755b7c5f18'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:08.436130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f59ec9365de4de284acad755b7c5f18'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:08.448285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f59ec9365de4de284acad755b7c5f18'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:08.456225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f59ec9365de4de284acad755b7c5f18'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:08.464590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f59ec9365de4de284acad755b7c5f18'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:08.470679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1231/2476] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:08 2022] DELETE /identity/v3/users/4f59ec9365de4de284acad755b7c5f18 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:08.475891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.476466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.476466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.495736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:08.496937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:08.526146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.559865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1246/2477] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:08.569550 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.571002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:08.606410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:08.631698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773b7d-0cfd-4e49-91e6-6a036457aaaf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.635418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.636043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.636322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.636595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.646688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.648133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef18937-76e9-4451-97e9-3b8e36ed0fbf service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:08.657917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1232/2478] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:08.929089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:08.930621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:08.943012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc8c8-bc7b-47fa-bcc5-569cbd1d64d4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:08.947725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:08.948392 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:08.948681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:08.948956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] PATH_INFO: `/v3/projects/1180260701324813ab68960aa4bdbe19` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:08.950226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1180260701324813ab68960aa4bdbe19)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:08.952159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.007409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1180260701324813ab68960aa4bdbe19', 'request_id': 'req-af8e5fbc-3994-46a8-ab32-7ce5453f515e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:09.015325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1180260701324813ab68960aa4bdbe19', 'request_id': 'req-af8e5fbc-3994-46a8-ab32-7ce5453f515e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:09.022603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1180260701324813ab68960aa4bdbe19', 'request_id': 'req-af8e5fbc-3994-46a8-ab32-7ce5453f515e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:09.033471 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] The token cache is being invalidate because project 1180260701324813ab68960aa4bdbe19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:09.033761 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:09.034450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-af8e5fbc-3994-46a8-ab32-7ce5453f515e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:09.035975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1247/2479] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:08 2022] DELETE /identity/v3/projects/1180260701324813ab68960aa4bdbe19 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:09.041538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.041829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.042140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.061505 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:09.062746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:09.124698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1233/2480] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:09.133086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.133373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.133653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.153962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:09.155179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:09.163083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1248/2481] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:09.167444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.177464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d436-ff9a-4791-8948-0c6be15d2f5d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.182253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.182924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.183211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.183490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.184205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:09.184516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.186176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.199046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1234/2482] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:09.203628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.214294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19396a72-6f59-4ac1-ad74-ef104973280e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.219036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.219665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.219946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.220223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.220906 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.222901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.253762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1249/2483] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:09.259337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.269861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c662368-549f-4426-b966-47827d2f9507 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.324777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.325491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.325655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.325953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.326614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.328539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.351212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1235/2484] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:09.355526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.365927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eabeb-171b-4d2e-8180-575a5364f00a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.370508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.371126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.371411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.371689 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.372386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.374065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.384542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1250/2485] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:09.388512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.399233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5818-ea9c-43f7-90e1-b58e0ba97910 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.404007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.404623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.404929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.405212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.405976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.407589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf88f1-b8d2-4459-a948-61a00811aca3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.418531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1236/2486] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:09.422762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.432893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc977ae-12b4-47dc-91e1-b1d7ee67da37 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.437635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.438348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.438634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.438910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] PATH_INFO: `/v3/projects/e979b76a9972479f8729dc8c26e0ce0e/users/1da2660b262e41b7a3fb2dc2c84a182f/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.454583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e979b76a9972479f8729dc8c26e0ce0e, user_id=1da2660b262e41b7a3fb2dc2c84a182f, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.457311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.472051 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1251/2487] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:09 2022] PUT /identity/v3/projects/e979b76a9972479f8729dc8c26e0ce0e/users/1da2660b262e41b7a3fb2dc2c84a182f/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:09.482774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.483048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.483321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.511795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] MFA Rules not processed for user `1da2660b262e41b7a3fb2dc2c84a182f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:09.512999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:09.541827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.578330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1237/2488] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:22:09.582812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.637382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:09.648857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2de24-29e1-4095-a6bb-10113bb8cab4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.655019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.655663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.655933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.656202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.656880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.658859 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.686633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1252/2489] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:09.691157 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.702620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b9a0-380d-414b-94ed-fb8f8c1c751d None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.760209 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.760843 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.761124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.761529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.762252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.764222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.789294 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1238/2490] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:09 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:09.793171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.804500 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16d871-4036-4298-877a-3f9b297a6866 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.809758 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.810464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.810737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.811003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.811727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.813347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.825154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1253/2491] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:09.829402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.841816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a385-e61e-4e63-adf8-03452fe1eb75 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.846704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.847380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.847664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.847942 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] PATH_INFO: `/v3/projects/493a88331f7a4a48ac85a6c4f6218362/users/58af67c5c286470eb50827ef9e838869/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.863188 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493a88331f7a4a48ac85a6c4f6218362, user_id=58af67c5c286470eb50827ef9e838869, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.866437 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.879848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1239/2492] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:09 2022] PUT /identity/v3/projects/493a88331f7a4a48ac85a6c4f6218362/users/58af67c5c286470eb50827ef9e838869/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:09.884353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.894459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fa88c-3640-46d5-b8b7-dd2374f7e589 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.952517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.953143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.953428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.953818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.954544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.956152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7d351-2e70-4dc4-9f2b-23c0f6c2f24f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:09.966785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1254/2493] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:09.970934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:09.982010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d6712-66cf-4e6a-b842-e7beef205051 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:09.987316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:09.987976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:09.988254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:09.988528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] PATH_INFO: `/v3/projects/493a88331f7a4a48ac85a6c4f6218362/users/58af67c5c286470eb50827ef9e838869/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:09.992419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] RBAC: Authorizing `identity:create_grant(project_id=493a88331f7a4a48ac85a6c4f6218362, user_id=58af67c5c286470eb50827ef9e838869, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:09.995143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.005779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1240/2494] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:09 2022] PUT /identity/v3/projects/493a88331f7a4a48ac85a6c4f6218362/users/58af67c5c286470eb50827ef9e838869/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:10.016839 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.017127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.017409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.046151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] MFA Rules not processed for user `58af67c5c286470eb50827ef9e838869`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:10.047375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:10.123712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1255/2495] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:10.140921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:10.142691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:10.199476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d262a-caca-4ed5-ad8a-1bd0710e937d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:10.203202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:10.203817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:10.204096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:10.204368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:10.237087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:10.261779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:10.263253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0694d5-5e82-4cee-9f95-b3b57c76ee59 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:10.272748 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1241/2496] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:12.813369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:12.859861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:12.874372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3131-60e1-4925-aba7-b82c0b57f970 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:12.878862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:12.879498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.879775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.880046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] PATH_INFO: `/v3/users/1da2660b262e41b7a3fb2dc2c84a182f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.882123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da2660b262e41b7a3fb2dc2c84a182f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:12.884035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:12.934426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da2660b262e41b7a3fb2dc2c84a182f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:12.945959 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da2660b262e41b7a3fb2dc2c84a182f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:12.952706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da2660b262e41b7a3fb2dc2c84a182f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:12.953378 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da2660b262e41b7a3fb2dc2c84a182f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:12.954016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da2660b262e41b7a3fb2dc2c84a182f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:12.967155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1256/2497] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:12 2022] DELETE /identity/v3/users/1da2660b262e41b7a3fb2dc2c84a182f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:12.971655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:12.972126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:12.972223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:12.992213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:12.994198 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.054197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1242/2498] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:13.063917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.065323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.122798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392ffa7-a2bd-4040-a4ab-a9f422b2420d admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.126476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.127094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.127379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.127657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.136752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.138220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.147752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1257/2499] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:13.415359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.416772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.429018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef822226-79dc-43ae-9fe4-4a5a0b238322 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.433947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.434725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.435010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.435289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] PATH_INFO: `/v3/projects/e979b76a9972479f8729dc8c26e0ce0e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.437037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e979b76a9972479f8729dc8c26e0ce0e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.439179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.487742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e979b76a9972479f8729dc8c26e0ce0e', 'request_id': 'req-50c2dd6d-82f8-469e-b4b1-64a433633b9a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.494729 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e979b76a9972479f8729dc8c26e0ce0e', 'request_id': 'req-50c2dd6d-82f8-469e-b4b1-64a433633b9a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.500936 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e979b76a9972479f8729dc8c26e0ce0e', 'request_id': 'req-50c2dd6d-82f8-469e-b4b1-64a433633b9a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.511280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] The token cache is being invalidate because project e979b76a9972479f8729dc8c26e0ce0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:13.511568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.512202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.513708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1243/2500] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:13 2022] DELETE /identity/v3/projects/e979b76a9972479f8729dc8c26e0ce0e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:13.517722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.527873 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da293a-e5d0-43fc-96aa-2d9657eabdd1 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:13.583876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:13.585052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:13.585337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:13.585615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] PATH_INFO: `/v3/users/58af67c5c286470eb50827ef9e838869` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:13.586818 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58af67c5c286470eb50827ef9e838869)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:13.589411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:13.602988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:13.637110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58af67c5c286470eb50827ef9e838869'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.650861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58af67c5c286470eb50827ef9e838869'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.658110 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58af67c5c286470eb50827ef9e838869'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.658815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58af67c5c286470eb50827ef9e838869'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.659415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e63ad81c-22f6-484e-be9a-7a3f22c34d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58af67c5c286470eb50827ef9e838869'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:13.671923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1258/2501] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:13 2022] DELETE /identity/v3/users/58af67c5c286470eb50827ef9e838869 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:13.941710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:13.943273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:13.957343 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2dd6d-82f8-469e-b4b1-64a433633b9a admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.012621 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.013272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.013519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.013792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] PATH_INFO: `/v3/projects/493a88331f7a4a48ac85a6c4f6218362` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.014954 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=493a88331f7a4a48ac85a6c4f6218362)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.016852 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.018264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.063286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493a88331f7a4a48ac85a6c4f6218362', 'request_id': 'req-827a5eab-36bf-403f-99ce-24919af4ff0f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:14.070625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493a88331f7a4a48ac85a6c4f6218362', 'request_id': 'req-827a5eab-36bf-403f-99ce-24919af4ff0f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:14.076871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493a88331f7a4a48ac85a6c4f6218362', 'request_id': 'req-827a5eab-36bf-403f-99ce-24919af4ff0f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:14.087322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] The token cache is being invalidate because project 493a88331f7a4a48ac85a6c4f6218362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:14.087612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:14.088240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-827a5eab-36bf-403f-99ce-24919af4ff0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:14.089785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1244/2502] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:13 2022] DELETE /identity/v3/projects/493a88331f7a4a48ac85a6c4f6218362 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:14.094657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.095143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.095276 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.114202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:14.115944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:14.172198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1259/2503] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:14.180756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.181041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.181350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.201203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:14.202463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:14.210860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1245/2504] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:14.214912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.224651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48fe3f-a036-41af-921e-5dd1f87d16d3 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.229502 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.230157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.230442 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.230715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.231412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:14.231718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.233351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.245352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1260/2505] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:14.249518 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.259326 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67ddd2-173c-43fe-ae14-0030914d27ea None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.264045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.264644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.264923 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.265194 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.265885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.267777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.293501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1246/2506] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:14.297629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.308025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65000dc-6488-4ed3-9c9e-5cd6055ae335 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.363066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.363677 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.363952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.364270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.364927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.367404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.388086 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1261/2507] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:14.392129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.402366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e59d-8271-4577-9362-ee1704ab58ee admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.406960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.407567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.407848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.408123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.408815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.410451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.420589 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1247/2508] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:14.424368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.434855 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38b08-e1f7-45ee-b792-329c4c53ece6 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.439498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.440096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.440373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.440644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.441335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.442927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b98c-e78d-4d51-97bf-79da5f80c5eb admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.444309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:14.455397 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1262/2509] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:14.459498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.469344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c38f26-889c-431f-a829-240758248f63 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.473985 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.474641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.475427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.475701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] PATH_INFO: `/v3/projects/29dc499a715749a58d47a13b93f6f2fb/users/0c3f0161741e48d0be671c6be1b495d2/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.489948 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29dc499a715749a58d47a13b93f6f2fb, user_id=0c3f0161741e48d0be671c6be1b495d2, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.493092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.505605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1248/2510] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:14 2022] PUT /identity/v3/projects/29dc499a715749a58d47a13b93f6f2fb/users/0c3f0161741e48d0be671c6be1b495d2/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:14.516476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.516766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.517039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.543942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] MFA Rules not processed for user `0c3f0161741e48d0be671c6be1b495d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:14.545135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:14.607673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1263/2511] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:14.619321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:14.620886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:14.676291 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41844787-a4f8-4ecd-9d62-c226585fd342 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:14.680610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:14.681424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:14.681712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:14.682014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:14.691776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:14.693342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636f63-1176-4790-9c29-a600648f4ec3 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:14.705815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1249/2512] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:17.245554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.305609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea85354-b7a2-4cc1-98fd-5617d29445b6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.310040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.310671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.310947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.311216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] PATH_INFO: `/v3/users/0c3f0161741e48d0be671c6be1b495d2` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.312358 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3f0161741e48d0be671c6be1b495d2)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.314310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.359459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3f0161741e48d0be671c6be1b495d2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.371776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3f0161741e48d0be671c6be1b495d2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.378577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3f0161741e48d0be671c6be1b495d2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.379246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3f0161741e48d0be671c6be1b495d2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.379853 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3f0161741e48d0be671c6be1b495d2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.394921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1264/2513] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:17 2022] DELETE /identity/v3/users/0c3f0161741e48d0be671c6be1b495d2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:17.399546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.399992 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.400088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.418842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:17.420037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:17.443302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.479108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1250/2514] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:17.488675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.490126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:17.497346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:17.548623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b398fa-5241-4b79-8865-6cbd78fe3f19 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.552264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.552929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.553708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.554019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.562993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.564430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffe138-cd24-4708-aebe-7077be6e83b9 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.573757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1265/2515] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:17.844280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:17.849524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:17.864087 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f10c0-c1c3-4ed2-af1a-e0b0503283c2 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:17.870125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:17.871213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.871733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.872184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] PATH_INFO: `/v3/projects/29dc499a715749a58d47a13b93f6f2fb` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:17.874286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=29dc499a715749a58d47a13b93f6f2fb)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:17.878093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:17.937781 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29dc499a715749a58d47a13b93f6f2fb', 'request_id': 'req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.953187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29dc499a715749a58d47a13b93f6f2fb', 'request_id': 'req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.963761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29dc499a715749a58d47a13b93f6f2fb', 'request_id': 'req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.978200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] The token cache is being invalidate because project 29dc499a715749a58d47a13b93f6f2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:17.978510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.979191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a77380dc-3a36-475a-9a0c-ffe6ca4e67fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:17.980738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1251/2516] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:17 2022] DELETE /identity/v3/projects/29dc499a715749a58d47a13b93f6f2fb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:17.985667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:17.986085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:17.986408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.006506 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:18.007739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.073313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1266/2517] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:18.081172 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.081465 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.081737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.102635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:18.103842 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.111643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1252/2518] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:18.115934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.126712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827e4d8-8f90-4391-9f3a-66eae1f72701 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.131764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.132987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.133262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.133532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.134270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:18.134690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.136320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.149178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1267/2519] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:18.153763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.163314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb28ca-d3b7-4d63-9443-69a4142750dd None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.167933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.168544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.168822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.169093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.169749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.171686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.200590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1253/2520] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:18 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:18.204950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.216060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae0ca-656d-4d03-9651-26fc754ac582 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.275986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.276664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.276994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.277267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.277966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.279917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.303120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1268/2521] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:18 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:18.306623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.317113 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84dde0-38dd-46a4-a232-b4a795aececf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.322398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.323024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.323307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.323635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.324282 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.325922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.336934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1254/2522] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:18.341122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.351267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e15a0-b2e1-4a83-b098-b71c8e590de6 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.355903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.356594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.356874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.357146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.357861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.359497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfff9c-4d9c-410c-bbf3-c1c6bbdb4681 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.370346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1269/2523] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:18.374557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.385480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5314bd-fcb7-4ae8-be43-f6913f8a688c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.390140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.390807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.391086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.391362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] PATH_INFO: `/v3/projects/32a3369fd12d4e6d9139ab4af7e6a6a7/users/3fc2d12913df4003b6a7c5637d638464/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.408011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a3369fd12d4e6d9139ab4af7e6a6a7, user_id=3fc2d12913df4003b6a7c5637d638464, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.410693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.424270 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1255/2524] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:18 2022] PUT /identity/v3/projects/32a3369fd12d4e6d9139ab4af7e6a6a7/users/3fc2d12913df4003b6a7c5637d638464/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:18.434578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.434865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.435136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.462078 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.465795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] MFA Rules not processed for user `3fc2d12913df4003b6a7c5637d638464`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:18.467706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.532477 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1270/2525] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:18.543801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.545470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.597999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.602342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807c1eb-23bc-4241-95ae-41c01806acb6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.607028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.607757 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.608004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.608462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.618052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.619542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0f53d-31e6-4d61-92f4-e83d6312628d service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.628397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1256/2526] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:18.653724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.716356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5d9ba-49e9-4e41-9f7b-90d7590b79cc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.720939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.721579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.721877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.722152 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] PATH_INFO: `/v3/users/3fc2d12913df4003b6a7c5637d638464` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.723330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc2d12913df4003b6a7c5637d638464)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.725264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.775538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc2d12913df4003b6a7c5637d638464'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:18.787478 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc2d12913df4003b6a7c5637d638464'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:18.794190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc2d12913df4003b6a7c5637d638464'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:18.794898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc2d12913df4003b6a7c5637d638464'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:18.795516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc2d12913df4003b6a7c5637d638464'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:18.808798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1271/2527] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:18 2022] DELETE /identity/v3/users/3fc2d12913df4003b6a7c5637d638464 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:18.813583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.813905 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.814182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.834228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:18.835530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.900339 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1257/2528] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:22:18.909802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:18.911256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:18.939627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:18.973902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2ee0c-d86b-4cf7-b715-a65681855e72 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:18.977671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:18.978351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:18.978631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:18.978901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:18.988049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:18.989479 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f9df6-089e-4e15-8ad0-07297df2fd96 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:18.999182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1272/2529] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:19.288254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.288631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.300667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c33b5e-c82b-433d-b828-b6fe20a6d6da None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.305958 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.306614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.306895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.307169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] PATH_INFO: `/v3/projects/32a3369fd12d4e6d9139ab4af7e6a6a7` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.308366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a3369fd12d4e6d9139ab4af7e6a6a7)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.310411 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.360775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a3369fd12d4e6d9139ab4af7e6a6a7', 'request_id': 'req-2891e4f5-d840-4303-9a9d-c5ae98f92662'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:19.368104 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a3369fd12d4e6d9139ab4af7e6a6a7', 'request_id': 'req-2891e4f5-d840-4303-9a9d-c5ae98f92662'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:19.374304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a3369fd12d4e6d9139ab4af7e6a6a7', 'request_id': 'req-2891e4f5-d840-4303-9a9d-c5ae98f92662'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:19.385063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] The token cache is being invalidate because project 32a3369fd12d4e6d9139ab4af7e6a6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:19.385349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:19.386029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2891e4f5-d840-4303-9a9d-c5ae98f92662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:19.387562 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1258/2530] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:19 2022] DELETE /identity/v3/projects/32a3369fd12d4e6d9139ab4af7e6a6a7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:19.392671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.393155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.393248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.412657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:19.413769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.477720 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1273/2531] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:19.486557 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.487069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.487185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.507668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:19.509768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.518164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1259/2532] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:19.522402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.532661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5cd3c-1312-43ac-a2c8-ec7c06790a6e None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.537293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.537937 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.538813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.539099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.539794 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:19.540096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.541688 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.553138 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1274/2533] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:19.557624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.567021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b46c64-e747-4460-aca1-15ee0638f60a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.571804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.572442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.572726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.573002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.573666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.575640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.603641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1260/2534] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:19.607903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.618282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634475eb-70d9-43cf-9966-64beb4b66cec admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.675414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.675951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.676228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.676499 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.677175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.679486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.701543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1275/2535] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:19.705595 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.716776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829e3f7-7d40-4a25-9c56-bcc58c931b50 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.721372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.722018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.722297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.722569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.723257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.724987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.735057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1261/2536] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:19.738967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.749134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b575437-3278-404e-9fe5-8153a553140b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.753708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.754560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.754712 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.754900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.755680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.757940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96c59f-dee0-4cc7-85e1-175b11aaba1f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.768635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1276/2537] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:19.772825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.783390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b69e6-8f41-4c14-94c3-2de0f6982d35 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:19.788041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:19.788749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.789030 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.789304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] PATH_INFO: `/v3/projects/b688d320376c4492b3ca97b95c5207a5/users/f56d3e6849ff4268a03ef67c8cf0f813/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.804556 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b688d320376c4492b3ca97b95c5207a5, user_id=f56d3e6849ff4268a03ef67c8cf0f813, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:19.807255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:19.822367 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1262/2538] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:19 2022] PUT /identity/v3/projects/b688d320376c4492b3ca97b95c5207a5/users/f56d3e6849ff4268a03ef67c8cf0f813/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:19.827614 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8fa8-0158-44e3-9f68-3b108b701fa6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:19.827900 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8fa8-0158-44e3-9f68-3b108b701fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:19.828179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8fa8-0158-44e3-9f68-3b108b701fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:19.848784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-7cdf8fa8-0158-44e3-9f68-3b108b701fa6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:19.850060 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7cdf8fa8-0158-44e3-9f68-3b108b701fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.879568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:19.926044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1277/2539] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:19.936260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:19.938018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:19.951740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:20.002712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982a64-4625-43b3-abd2-1d54aa7395c5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:20.006560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:20.007772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:20.008049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:20.008317 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:20.017395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:20.018848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:20.028422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1263/2540] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:24.457810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.458894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.458894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.491897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] MFA Rules not processed for user `f56d3e6849ff4268a03ef67c8cf0f813`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:24.493211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:24.588732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1278/2541] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:24.593397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.607347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125601f5-5442-4853-b097-3fc40bba82a2 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.616804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.618073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.619016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.619016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.620149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.623845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.674877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1264/2542] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:24 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:24.678628 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.693551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00232a2e-e998-40d9-9793-0525b7e8ab79 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.777176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.778464 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.778927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.779377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.780577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.784282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.821631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1279/2543] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:24 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:24.822633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.839359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dc33-d0ba-42b7-a3c9-26bc3178d1fb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.852232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.853135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.853643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.854212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.855071 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.858082 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.875643 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1265/2544] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:24.883123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.900480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd6d69-eaea-44ef-a577-1b0f7d31fde3 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:24.905620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:24.906469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:24.906843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:24.907194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] PATH_INFO: `/v3/projects/fe092e0cbbe4425795519f6c4e9c976d/users/cf2b6a493cbf474bb23d54ea1fea4f74/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:24.927988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe092e0cbbe4425795519f6c4e9c976d, user_id=cf2b6a493cbf474bb23d54ea1fea4f74, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:24.930969 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:24.954825 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1280/2545] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:24 2022] PUT /identity/v3/projects/fe092e0cbbe4425795519f6c4e9c976d/users/cf2b6a493cbf474bb23d54ea1fea4f74/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:24.956054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:24.969535 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48537d8-3e20-4480-9c7d-ab5aea904670 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.039066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.039703 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.039983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.040258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.040961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.042620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.054094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1266/2546] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:25.058755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:25.069964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d9508-2ac9-4940-8cdf-f9f8f605bc32 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:25.074763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:25.075535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:25.075819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:25.076101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] PATH_INFO: `/v3/projects/fe092e0cbbe4425795519f6c4e9c976d/users/cf2b6a493cbf474bb23d54ea1fea4f74/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:25.078211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe092e0cbbe4425795519f6c4e9c976d, user_id=cf2b6a493cbf474bb23d54ea1fea4f74, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:25.081192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256131f-6848-4ba5-a0dd-0503f4d9e3a3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:25.095278 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1281/2547] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:25 2022] PUT /identity/v3/projects/fe092e0cbbe4425795519f6c4e9c976d/users/cf2b6a493cbf474bb23d54ea1fea4f74/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:27.249676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:27.251753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:27.387717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a6e12-0454-45ce-9b4b-dce86d8bca80 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:27.415074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:27.539403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:27.539881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:27.540163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:27.540436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:27.542564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:27.705935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:27.707813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:27.757252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1267/2548] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:30.966293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:30.966902 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:30.967241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.026832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:31.035551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] MFA Rules not processed for user `cf2b6a493cbf474bb23d54ea1fea4f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:31.037371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:31.171626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:31.178284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1282/2549] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:31.201025 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:31.224224 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2525a2-d73e-4011-b9d7-aa14df93bb16 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:31.232260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.233390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.233865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.234240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.261548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c668-28f2-457f-ad2d-439d29cc4745 service nova] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:31.268289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:31.269155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:31.269541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:31.269943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:31.272413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:31.295720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.298417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.313487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1268/2550] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:22:31.318517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:31.321114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:31.331952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1283/2551] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:22:41.308406 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=88 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:41.326815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=56 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:49.591961 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:49.593395 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:49.603877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17501-67c3-490d-9373-0709fb413b60 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:49.607558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:49.608183 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:49.608459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:49.608731 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:49.610534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:49.675180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:49.676859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:49.686816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1269/2552] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:50.363043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:50.373172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0c57f-2373-46f0-b231-edcef8161c8d service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:50.376890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:50.377522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:50.377806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:50.378120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:50.380020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:50.392529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:50.394017 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:50.404637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1284/2553] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:52.877677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:52.879755 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.002814 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.007139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.023131 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d16a7-4c8d-41b1-9303-1656c7ae639c service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.028139 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.028798 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.029248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.029697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] PATH_INFO: `/v3/users/a6f5e471a7ac4218a35ffe1840d91575` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.031074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f5e471a7ac4218a35ffe1840d91575)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:53.033047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:53.046136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf158e-8d2d-431b-97b5-831794969355 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.051910 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.052655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.052959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.053239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.055758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.068683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:53.070653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:53.082991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1270/2554] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:53.096099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f5e471a7ac4218a35ffe1840d91575'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.109955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f5e471a7ac4218a35ffe1840d91575'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.116675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f5e471a7ac4218a35ffe1840d91575'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.117979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f5e471a7ac4218a35ffe1840d91575'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.118678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-61851844-84f0-4175-b223-f21fe17c5b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f5e471a7ac4218a35ffe1840d91575'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.137528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1285/2555] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:53 2022] DELETE /identity/v3/users/a6f5e471a7ac4218a35ffe1840d91575 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:53.359381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.373654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb7ff6-381a-476d-8010-beba6318012c service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.483215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.483866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.484149 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.484425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] PATH_INFO: `/v3/projects/6e862416d3ca4d019cd0dd62f439038c` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.485734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e862416d3ca4d019cd0dd62f439038c)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:53.487959 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:53.533174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:53.559647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e862416d3ca4d019cd0dd62f439038c', 'request_id': 'req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.574389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e862416d3ca4d019cd0dd62f439038c', 'request_id': 'req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.575567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffffd2-0d91-44e9-be34-014a2fe1e7e4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.575932 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffffd2-0d91-44e9-be34-014a2fe1e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.576284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffffd2-0d91-44e9-be34-014a2fe1e7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.584439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e862416d3ca4d019cd0dd62f439038c', 'request_id': 'req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.598808 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] The token cache is being invalidate because project 6e862416d3ca4d019cd0dd62f439038c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:53.599394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.600587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f7c8d4d2-88f3-4ccc-b9c2-d0b0415464f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:53.601419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6dffffd2-0d91-44e9-be34-014a2fe1e7e4 None None] MFA Rules not processed for user `f56d3e6849ff4268a03ef67c8cf0f813`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:53.603293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6dffffd2-0d91-44e9-be34-014a2fe1e7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.603507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1271/2556] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:53 2022] DELETE /identity/v3/projects/6e862416d3ca4d019cd0dd62f439038c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:53.610549 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.610920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.611261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.638178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:53.639608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.740315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1286/2557] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:22:53.753966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1272/2558] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:22:53.761402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.761715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.762119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.796259 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:53.797885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:53.824554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1287/2559] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:53.828758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.841971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746c387-954b-4759-83ba-9e2d8796803e None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.847335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.848308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.848308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.848645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.849448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:53.849802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:53.851561 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:53.864940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1273/2560] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:22:53.869176 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.878795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772657-53a5-406b-9b68-abbe9fac7ebc None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:53.887326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:53.888330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:53.888827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:53.889190 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:53.890642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:53.893112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:53.944141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1288/2561] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:53.948509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:53.962393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1dfa9-598a-4a29-bc20-8e8848f9013b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.029500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.030088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.030377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.030659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.031328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.033531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.068821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1274/2562] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:53 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:54.075223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.103709 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831637f-cef1-4287-9d6c-b47c4c5e8167 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.110169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.111705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.111999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.112282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.117275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.119617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.135991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1289/2563] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:54.140647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.161463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747b3bb-4660-4e98-a9bc-23d137006c6b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.168268 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.168914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.169222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.169585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.170297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.172077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809992-1e56-442e-8496-4ac20f2e0fb0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.188059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1275/2564] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:54.192235 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.203928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d368e2-378d-4d57-aa09-8f900dd8fb61 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.210615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.212023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.212409 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.212944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] PATH_INFO: `/v3/projects/9caa7ccea6b747fabd869dc147cae279/users/8e73e37fe3204151993741fbfb886332/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.236648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9caa7ccea6b747fabd869dc147cae279, user_id=8e73e37fe3204151993741fbfb886332, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.239408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.248448 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.254011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1290/2565] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:54 2022] PUT /identity/v3/projects/9caa7ccea6b747fabd869dc147cae279/users/8e73e37fe3204151993741fbfb886332/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:22:54.265492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.265813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.266186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.297258 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] MFA Rules not processed for user `8e73e37fe3204151993741fbfb886332`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:54.298544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:54.312360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:54.323647 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.326159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:54.389641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1276/2566] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:54.402072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.404124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:54.407161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d780e-dd02-42f9-9e5c-f028cc262631 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.412135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.412810 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.413091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.413469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.415273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:54.426948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.428443 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.438427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1291/2567] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:54.471472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47277c-93f2-40d0-a686-219d860e0d1f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.476163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.476887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.477152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.477401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.487496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.489057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.500070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1277/2568] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:54.716795 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:54.718843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:54.817604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de6d62-5757-4e94-adb7-cc814b629544 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:54.825336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:54.827087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:54.827524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:54.827807 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] PATH_INFO: `/v3/users/e0c1ed47784c46baa7ddc4abdfc54bbb` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:54.829923 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c1ed47784c46baa7ddc4abdfc54bbb)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:54.832748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:54.896856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c1ed47784c46baa7ddc4abdfc54bbb'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:54.911123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c1ed47784c46baa7ddc4abdfc54bbb'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:54.920379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c1ed47784c46baa7ddc4abdfc54bbb'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:54.921439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c1ed47784c46baa7ddc4abdfc54bbb'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:54.922270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c1ed47784c46baa7ddc4abdfc54bbb'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:54.937425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1292/2569] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:54 2022] DELETE /identity/v3/users/e0c1ed47784c46baa7ddc4abdfc54bbb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:55.128217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.157606 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d856-c6d0-4d17-9579-8acd93e59e3f service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.180718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.182609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.213874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.230375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.231118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.231404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.231704 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] PATH_INFO: `/v3/projects/c957cfe3391147daab9fc1e59b414a3e` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.234123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c957cfe3391147daab9fc1e59b414a3e)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.236235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.267676 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb4d4-e18d-4b51-8625-77d3e6195e51 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.273263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.274704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.274988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.275264 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.278142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.284486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c957cfe3391147daab9fc1e59b414a3e', 'request_id': 'req-01333808-9d6a-407d-8c41-7db992181bb5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:55.288243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.291655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.293318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c957cfe3391147daab9fc1e59b414a3e', 'request_id': 'req-01333808-9d6a-407d-8c41-7db992181bb5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:55.293509 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349fb88-c176-4a8b-b427-74377e6fd228 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.302869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c957cfe3391147daab9fc1e59b414a3e', 'request_id': 'req-01333808-9d6a-407d-8c41-7db992181bb5'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:55.304124 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1293/2570] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:22:55.314440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] The token cache is being invalidate because project c957cfe3391147daab9fc1e59b414a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:22:55.314728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:55.315405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-01333808-9d6a-407d-8c41-7db992181bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:55.316953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1278/2571] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:22:55 2022] DELETE /identity/v3/projects/c957cfe3391147daab9fc1e59b414a3e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:55.321889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.322185 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.322461 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.342518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:55.343706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.412822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1294/2572] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:55.422580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.422890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.423167 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.443965 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:55.445239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.453337 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1279/2573] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:55.457806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.469094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9bda-ab19-4e54-b7e8-0f2dce6f0c2a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.474009 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.474631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.474914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.475203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.475895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:22:55.476279 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.478653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.490722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1295/2574] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:22:55.495049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.505259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74964bb-5cae-40ff-9cf7-591348c3463c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.516983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.517618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.517934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.518212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.518892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.521123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.549743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1280/2575] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:55 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:55.554573 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.565320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c349-9ecf-4c41-9b05-dbbf9070f6b6 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.634770 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.635404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.635821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.636103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.636803 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.638957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.660796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1296/2576] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:55 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:55.665259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.677073 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0761707-dc9a-45b9-a96b-60d733e48c0c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.685240 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.685987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.686272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.686568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.687328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.688974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.702108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1281/2577] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:55.706072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.716996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9d26f-728b-419c-b094-6a05bed50720 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.722291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.722981 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.723263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.723539 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.724388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.726056 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f797-528c-4f1b-8e2b-4f799f7930fe admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.737618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1297/2578] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:55.741866 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.752184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5681-652d-423c-a147-65b0e3d9c708 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.757033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.757804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.758839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.759162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] PATH_INFO: `/v3/projects/8143761879274c36a2ec81932c3f6a16/users/0dddbf1e74054bdbabd30160ae500a47/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.775500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8143761879274c36a2ec81932c3f6a16, user_id=0dddbf1e74054bdbabd30160ae500a47, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.782408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:55.798016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1282/2579] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:55 2022] PUT /identity/v3/projects/8143761879274c36a2ec81932c3f6a16/users/0dddbf1e74054bdbabd30160ae500a47/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:55.803024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768dab-d557-4895-864c-a00b9a9dfce5 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.803311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768dab-d557-4895-864c-a00b9a9dfce5 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.803587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768dab-d557-4895-864c-a00b9a9dfce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.823686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-bc768dab-d557-4895-864c-a00b9a9dfce5 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:55.824985 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-bc768dab-d557-4895-864c-a00b9a9dfce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.837931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:55.890244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1298/2580] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:55.900953 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:55.902986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:55.973312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3cd-3684-448a-8f73-251ea6e3fc3f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:55.977232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:55.978094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:55.978249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:55.978507 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:55.991563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:55.993679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:56.004292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1283/2581] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:22:56.783196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.783805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.784316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.837652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] MFA Rules not processed for user `0dddbf1e74054bdbabd30160ae500a47`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:56.839425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:56.927558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1299/2582] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:56.932141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:56.942658 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46b5e-f8af-4725-9f29-ed1684db716a service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:56.947533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:56.948448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:56.948798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:56.949137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:56.950043 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:56.952059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:56.985682 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1284/2583] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:22:56 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:22:56.990365 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.003951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93455903-3c1e-40f8-ae9d-f50acd9f39c8 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.072729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.073374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.073656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.073978 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.074681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.076742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.104390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1300/2584] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:22:56 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:22:57.108344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.122600 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74254-963f-443d-99cc-48869a1c5523 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.127580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.128211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.128493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.128769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.129466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.131096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.142771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1285/2585] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:57.147632 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.158725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77d040-30f2-4560-ab19-decc95e33b64 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.163806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.164523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.164812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.165094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] PATH_INFO: `/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7/users/22aa5e29ce0542c68caba26c89efbb9f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.185503 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] RBAC: Authorizing `identity:create_grant(project_id=584eef3a91f443e2ac0e103f7ada1ad7, user_id=22aa5e29ce0542c68caba26c89efbb9f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.188332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.204163 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1301/2586] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:57 2022] PUT /identity/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7/users/22aa5e29ce0542c68caba26c89efbb9f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:57.208185 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.219611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b88148-5dac-42d4-a6b3-c88800fa558e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.271420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:57.291663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.292397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.292750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.293128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.293969 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.295687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be3b5c-d998-4355-a14b-58e72e59265c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.307743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1286/2587] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:22:57.311785 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:57.326386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9eb8-5307-4b3b-8e66-ddae05b9708b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:57.333549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:57.334591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:57.334876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:57.335156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] PATH_INFO: `/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7/users/22aa5e29ce0542c68caba26c89efbb9f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:57.338120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584eef3a91f443e2ac0e103f7ada1ad7, user_id=22aa5e29ce0542c68caba26c89efbb9f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:57.341252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:57.348321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:22:57.358407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1302/2588] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:22:57 2022] PUT /identity/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7/users/22aa5e29ce0542c68caba26c89efbb9f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:22:58.393248 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:58.393721 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:58.393814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:58.431824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] MFA Rules not processed for user `22aa5e29ce0542c68caba26c89efbb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:22:58.433329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:22:58.550610 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1287/2589] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:22:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:22:59.790686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:22:59.872808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd38d1-27fb-4216-9b7e-27357faa3fb2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:22:59.877765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:22:59.878544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:22:59.878894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:22:59.879238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] PATH_INFO: `/v3/users/0dddbf1e74054bdbabd30160ae500a47` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:22:59.880510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dddbf1e74054bdbabd30160ae500a47)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:22:59.882637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:22:59.939258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dddbf1e74054bdbabd30160ae500a47'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:59.953954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dddbf1e74054bdbabd30160ae500a47'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:59.965726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dddbf1e74054bdbabd30160ae500a47'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:59.966595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dddbf1e74054bdbabd30160ae500a47'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:59.967491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dddbf1e74054bdbabd30160ae500a47'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:22:59.985710 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1303/2590] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:22:59 2022] DELETE /identity/v3/users/0dddbf1e74054bdbabd30160ae500a47 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:00.452363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:00.465813 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db55d-4345-42bf-a732-2da7d1eb5de4 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:00.542698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:00.543530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.543819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.544101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] PATH_INFO: `/v3/projects/8143761879274c36a2ec81932c3f6a16` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.545336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8143761879274c36a2ec81932c3f6a16)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:00.547305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:00.592463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8143761879274c36a2ec81932c3f6a16', 'request_id': 'req-0714b89d-2477-4b52-9eff-3f8fe85c2778'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.600777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8143761879274c36a2ec81932c3f6a16', 'request_id': 'req-0714b89d-2477-4b52-9eff-3f8fe85c2778'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.607977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8143761879274c36a2ec81932c3f6a16', 'request_id': 'req-0714b89d-2477-4b52-9eff-3f8fe85c2778'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.614993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:00.619882 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] The token cache is being invalidate because project 8143761879274c36a2ec81932c3f6a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:00.620383 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.621530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.623349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1288/2591] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:00 2022] DELETE /identity/v3/projects/8143761879274c36a2ec81932c3f6a16 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:00.627496 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:00.628950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:00.642345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a4f64-a775-4113-beb8-88c7fa39685e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:00.703594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:00.704391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:00.704773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:00.705143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] PATH_INFO: `/v3/users/22aa5e29ce0542c68caba26c89efbb9f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:00.706554 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22aa5e29ce0542c68caba26c89efbb9f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:00.708601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:00.731782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:00.761778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22aa5e29ce0542c68caba26c89efbb9f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.774071 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22aa5e29ce0542c68caba26c89efbb9f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.781193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22aa5e29ce0542c68caba26c89efbb9f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.781957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22aa5e29ce0542c68caba26c89efbb9f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.782568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4fdae50e-17cf-461d-b0e9-f3110d73e831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22aa5e29ce0542c68caba26c89efbb9f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:00.796613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1304/2592] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:00 2022] DELETE /identity/v3/users/22aa5e29ce0542c68caba26c89efbb9f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:00.983379 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:00.984861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:00.998052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b89d-2477-4b52-9eff-3f8fe85c2778 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.056109 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.056667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.056994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.057267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] PATH_INFO: `/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.058446 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584eef3a91f443e2ac0e103f7ada1ad7)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.060440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.108293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584eef3a91f443e2ac0e103f7ada1ad7', 'request_id': 'req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:01.118987 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584eef3a91f443e2ac0e103f7ada1ad7', 'request_id': 'req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:01.125713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584eef3a91f443e2ac0e103f7ada1ad7', 'request_id': 'req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:01.139843 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] The token cache is being invalidate because project 584eef3a91f443e2ac0e103f7ada1ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:01.140513 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:01.140799 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-c34a4ae4-bd6e-4d4e-87a2-65a2e02c9f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:01.142578 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1289/2593] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:00 2022] DELETE /identity/v3/projects/584eef3a91f443e2ac0e103f7ada1ad7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:01.147813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.148102 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.150067 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.169867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:01.171249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:01.234151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1305/2594] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:01.253178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.253178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.253178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.271698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:01.272924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:01.280972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1290/2595] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:01.285281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.295134 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db3c13-dda6-4f95-b871-e3f042c33c08 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.299773 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.300398 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.300682 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.300958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.301661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:01.302007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.303610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.316217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1306/2596] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:01.321277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.330824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d9753-165c-436c-8677-895650ff3d97 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.335735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.336372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.336651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.336925 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.337620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.339627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.366909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1291/2597] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:01.371704 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.383928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603ab28-07d3-48cd-98ad-6c86b1dc9916 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.451965 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.452616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.452950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.453233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.455046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.456617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.479977 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1307/2598] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:01.484603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.496035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ae23-c94e-46a0-9a4b-36a6de6c08a3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.502512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.503146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.503425 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.503705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.504565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.506266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.520648 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1292/2599] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:01.525495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.538103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835286-6004-4d1f-a9ac-7dec307125a4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.543896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.544612 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.544967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.545311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.546054 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.547680 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cac9c-af0d-4396-a422-e56877c71812 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.562035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1308/2600] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:01.566943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.579217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd1794-1687-4906-8e1e-32759783d022 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.585208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.585907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.586352 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.586707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] PATH_INFO: `/v3/projects/6bb18b2d219347f99638d5f93ea4e2cd/users/44027884ef6a4175845ceb7bc05467f8/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.603306 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb18b2d219347f99638d5f93ea4e2cd, user_id=44027884ef6a4175845ceb7bc05467f8, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.606486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.620296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1293/2601] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:01 2022] PUT /identity/v3/projects/6bb18b2d219347f99638d5f93ea4e2cd/users/44027884ef6a4175845ceb7bc05467f8/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:01.630690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.630984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.631261 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.640374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:01.661528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] MFA Rules not processed for user `44027884ef6a4175845ceb7bc05467f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:01.662781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:01.729331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1309/2602] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:01.734013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.765298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:01.800056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d809f9-0e32-469d-9bec-33b1410359a3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.804745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.805363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.805644 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.805949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.806618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.808514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.835136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1294/2603] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:01.839930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.850531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452ea77-0fcf-45dc-be12-3debcdf8dc46 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.908962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.909601 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.909905 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.910180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.911252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.913039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.938236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1310/2604] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:01 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:01.942222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.953135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fcc03-aa96-4134-999d-18c8f7b34731 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.957698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.958333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.958613 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.958879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:01.959822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:01.961445 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:01.971722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1295/2605] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:01.975670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:01.985911 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bb7d6-134a-49dc-81a9-33b2f862fc1f admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:01.990549 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:01.991206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:01.991480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:01.991753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] PATH_INFO: `/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c/users/51cbbef751324dcaa235c5f0a35b876b/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.006936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b10d3f783c44ddbebcf3828ae3322c, user_id=51cbbef751324dcaa235c5f0a35b876b, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:02.010319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:02.022749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1311/2606] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:01 2022] PUT /identity/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c/users/51cbbef751324dcaa235c5f0a35b876b/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:02.026697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:02.036911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ecf4d-f10b-4bf7-bb90-8c076ebe3dd0 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:02.091653 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:02.092230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:02.092510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:02.092791 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.093475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:02.095101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7007-7638-43b8-a862-962b65d168aa admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:02.104520 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1296/2607] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:02.108596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:02.119361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1338f1e-e9db-4992-90af-e7e8bca6b206 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:02.124081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:02.124750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:02.125062 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:02.125344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] PATH_INFO: `/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c/users/51cbbef751324dcaa235c5f0a35b876b/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.127408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b10d3f783c44ddbebcf3828ae3322c, user_id=51cbbef751324dcaa235c5f0a35b876b, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:02.130115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:02.140605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1312/2608] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:02 2022] PUT /identity/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c/users/51cbbef751324dcaa235c5f0a35b876b/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:02.152531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:02.152824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:02.153103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.181543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] MFA Rules not processed for user `51cbbef751324dcaa235c5f0a35b876b`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:02.182788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:02.259163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1297/2609] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:02.270846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:02.272665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:02.338394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:02.348842 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf271-e61f-4d7f-8d4a-1e7cbfbacfe0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:02.353691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:02.354379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:02.354662 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:02.354955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.365889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:02.367955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:02.378300 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1313/2610] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:02.460853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:02.470597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14c8af-3bd3-4972-87ab-123240d69680 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:02.474154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:02.474768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:02.475046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:02.475320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:02.495652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:02.535742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:02.537189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c8254-0425-4c17-a5d9-f86b9298a7d5 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:02.547137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1298/2611] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:03.438537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:03.439975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:03.512245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a120-e298-4526-8f79-1a664214e85f service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:03.516763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:03.517405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.517683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.517990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] PATH_INFO: `/v3/users/8e73e37fe3204151993741fbfb886332` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.519174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e73e37fe3204151993741fbfb886332)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:03.521087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:03.566865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e73e37fe3204151993741fbfb886332'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:03.579273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e73e37fe3204151993741fbfb886332'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:03.587045 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e73e37fe3204151993741fbfb886332'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:03.587741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e73e37fe3204151993741fbfb886332'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:03.588334 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e73e37fe3204151993741fbfb886332'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:03.601402 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1314/2612] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:03 2022] DELETE /identity/v3/users/8e73e37fe3204151993741fbfb886332 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:03.606580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.607099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.607200 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.626153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:03.627370 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:03.690981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1299/2613] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:03.701243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:03.702729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:03.764828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ffb0-41dc-4d7a-941e-5efc5dba0f3e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:03.770815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:03.771457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:03.771856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:03.771998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:03.783332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:03.784793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ef192-4612-47af-9680-48fa0b8486b5 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:03.796909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1315/2614] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:04.134547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.136579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:04.159630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0dbc-4109-4c39-847a-4a525c466ebf None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.165386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.166094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.166377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.166656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] PATH_INFO: `/v3/projects/9caa7ccea6b747fabd869dc147cae279` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.167945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9caa7ccea6b747fabd869dc147cae279)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.169968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.233543 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9caa7ccea6b747fabd869dc147cae279', 'request_id': 'req-20423237-36a3-47f5-ac23-36a7e6e7878a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:04.241450 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9caa7ccea6b747fabd869dc147cae279', 'request_id': 'req-20423237-36a3-47f5-ac23-36a7e6e7878a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:04.248806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9caa7ccea6b747fabd869dc147cae279', 'request_id': 'req-20423237-36a3-47f5-ac23-36a7e6e7878a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:04.260530 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] The token cache is being invalidate because project 9caa7ccea6b747fabd869dc147cae279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:04.260886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:04.261591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-20423237-36a3-47f5-ac23-36a7e6e7878a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:04.263239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1300/2615] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:04 2022] DELETE /identity/v3/projects/9caa7ccea6b747fabd869dc147cae279 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:04.268129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.268420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.268699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.288210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:04.289458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:04.294939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:04.351494 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1316/2616] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:04.359280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.359590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.359864 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.383627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:04.384856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:04.392887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1301/2617] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:04.397158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.406939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600bd82-d22d-4128-8f02-5e765e0b11f6 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.411475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.412082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.412359 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.412651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.413347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:04.413650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.415304 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.426658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1317/2618] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:04.431488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.440812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6136c8-6384-4802-a5e8-927c9505c2d0 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.446182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.446796 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.447076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.447350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.448008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.449934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.476769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1302/2619] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:04.480778 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.490685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695661b4-a720-4eb4-88d9-59af28e63015 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.545597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.546270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.546595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.546946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.547653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.549585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.571361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1318/2620] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:04.575453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.586022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be9cd2-d03e-459e-a753-ce84bc45b8d6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.592528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.593140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.593418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.593690 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.594417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.596007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.606346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1303/2621] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:04.610306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.621027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593e0b-d58b-4f02-8967-43cde7c46589 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.625727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.626384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.626670 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.626952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.627649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.629251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e563a-4b35-4f35-9316-b15a548566ba admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.641331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1319/2622] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:04.645650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.652182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:04.656179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffceafb-9f65-486a-a962-f002a003f44b admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.662037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.662945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.663387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.663838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] PATH_INFO: `/v3/projects/cf828b4e0cf644139fc076562bb66ba9/users/f4b2fb04a1eb44ebb8f09c28aab6d226/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.678919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf828b4e0cf644139fc076562bb66ba9, user_id=f4b2fb04a1eb44ebb8f09c28aab6d226, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.682243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.694878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1304/2623] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:04 2022] PUT /identity/v3/projects/cf828b4e0cf644139fc076562bb66ba9/users/f4b2fb04a1eb44ebb8f09c28aab6d226/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:04.709459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.709747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.710059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.739004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] MFA Rules not processed for user `f4b2fb04a1eb44ebb8f09c28aab6d226`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:04.740167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:04.809123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1320/2624] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:04.813751 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.881344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c75f-bf9b-497c-b3e1-84e49ab73a31 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.886150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.886771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.887313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.887756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.888485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.890431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:04.918666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1305/2625] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:04.923253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:04.935322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe8ec0-6765-4b1b-bda1-466d62d12c68 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:04.939192 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:04.993021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:04.993562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:04.993871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:04.994142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:04.994824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:04.996786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.021401 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1321/2626] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:04 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:05.026175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.038100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b0f9-fddf-45cb-80ca-2f9002dbf4de admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.044334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.045000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.045246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.045642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.046389 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.048041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.064286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1306/2627] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:05.066191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.076425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff37097-2710-4bf2-8818-e4e06d39f088 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.081059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.081726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.082044 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.082320 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] PATH_INFO: `/v3/projects/0392cedb0ff64712aba8d8a74e6afad6/users/03579b1f16884778b31ed28bbc883b6f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.098483 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0392cedb0ff64712aba8d8a74e6afad6, user_id=03579b1f16884778b31ed28bbc883b6f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.101164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.117619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1322/2628] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:05 2022] PUT /identity/v3/projects/0392cedb0ff64712aba8d8a74e6afad6/users/03579b1f16884778b31ed28bbc883b6f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:05.121475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.131947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68dfcc-4ec1-446e-9c2f-94b2ed219535 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.196408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.197825 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.198086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.198372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.199155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.200793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2c5d5-d385-4fae-8fc2-e4a08146c2de admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.216652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1307/2629] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:05.220884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.232446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4da4b9-de2c-4d8d-a501-c0d2f0b8f2ec admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.237210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.237989 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.238270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.238546 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] PATH_INFO: `/v3/projects/0392cedb0ff64712aba8d8a74e6afad6/users/03579b1f16884778b31ed28bbc883b6f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.240565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0392cedb0ff64712aba8d8a74e6afad6, user_id=03579b1f16884778b31ed28bbc883b6f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.243486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.254336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1323/2630] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:05 2022] PUT /identity/v3/projects/0392cedb0ff64712aba8d8a74e6afad6/users/03579b1f16884778b31ed28bbc883b6f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:05.266636 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.267037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.267310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.302394 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] MFA Rules not processed for user `03579b1f16884778b31ed28bbc883b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:05.303609 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:05.346390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:05.377977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1308/2631] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:05.390087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.391874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:05.454619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125fd3c-dad4-4f2c-ae88-184ffc8d0465 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.458727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.459645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.459921 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.460194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.479323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.481430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551119c5-d420-4ee4-9d82-b5f39f0271ed service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:05.492561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1324/2632] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:05.924374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:05.987056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ca544-3d68-46d5-92d7-778db98a9646 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:05.991552 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:05.992187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:05.992968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:05.993243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] PATH_INFO: `/v3/users/f4b2fb04a1eb44ebb8f09c28aab6d226` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:05.994430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b2fb04a1eb44ebb8f09c28aab6d226)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:05.996515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.043055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b2fb04a1eb44ebb8f09c28aab6d226'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.043705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b2fb04a1eb44ebb8f09c28aab6d226'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.057162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b2fb04a1eb44ebb8f09c28aab6d226'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.063867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b2fb04a1eb44ebb8f09c28aab6d226'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.071458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b2fb04a1eb44ebb8f09c28aab6d226'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.077244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1309/2633] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:05 2022] DELETE /identity/v3/users/f4b2fb04a1eb44ebb8f09c28aab6d226 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:06.081903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.082196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.082476 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.102219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:06.103433 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:06.163971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1325/2634] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:06.174275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.175676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:06.238173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6107-58f5-429b-a42c-50b73e3f22eb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.243788 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.244484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.244846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.245236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.255807 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.257163 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.261528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:06.272854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1310/2635] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:06.645706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.647470 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:06.656360 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:06.662912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fb38f-5c61-46cb-8c65-ba118fbf6d49 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.669002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.669701 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.670010 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.670294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] PATH_INFO: `/v3/projects/cf828b4e0cf644139fc076562bb66ba9` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.671473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf828b4e0cf644139fc076562bb66ba9)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.673490 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.722827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf828b4e0cf644139fc076562bb66ba9', 'request_id': 'req-05cac763-83d6-42ce-b943-e099fd4f39b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.731065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf828b4e0cf644139fc076562bb66ba9', 'request_id': 'req-05cac763-83d6-42ce-b943-e099fd4f39b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.737610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf828b4e0cf644139fc076562bb66ba9', 'request_id': 'req-05cac763-83d6-42ce-b943-e099fd4f39b1'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.748329 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] The token cache is being invalidate because project cf828b4e0cf644139fc076562bb66ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:06.748615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.749257 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.750802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1326/2636] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:06 2022] DELETE /identity/v3/projects/cf828b4e0cf644139fc076562bb66ba9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:06.754826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:06.765514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebcd-4393-42d2-baae-8cf827e3f544 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:06.825664 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:06.826341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:06.826623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:06.826894 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] PATH_INFO: `/v3/users/03579b1f16884778b31ed28bbc883b6f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:06.828062 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03579b1f16884778b31ed28bbc883b6f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:06.830039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:06.879041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03579b1f16884778b31ed28bbc883b6f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.879744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03579b1f16884778b31ed28bbc883b6f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.891960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03579b1f16884778b31ed28bbc883b6f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.900340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03579b1f16884778b31ed28bbc883b6f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.907224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-49ce98c0-4a68-40dc-b972-e7e855e4a531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03579b1f16884778b31ed28bbc883b6f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:06.913046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1311/2637] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:06 2022] DELETE /identity/v3/users/03579b1f16884778b31ed28bbc883b6f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:07.188241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.190284 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:07.205775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac763-83d6-42ce-b943-e099fd4f39b1 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.268125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.268865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.269150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.269432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] PATH_INFO: `/v3/projects/0392cedb0ff64712aba8d8a74e6afad6` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.270636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0392cedb0ff64712aba8d8a74e6afad6)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.272566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.320076 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0392cedb0ff64712aba8d8a74e6afad6', 'request_id': 'req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:07.328260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0392cedb0ff64712aba8d8a74e6afad6', 'request_id': 'req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:07.334743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0392cedb0ff64712aba8d8a74e6afad6', 'request_id': 'req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:07.345311 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] The token cache is being invalidate because project 0392cedb0ff64712aba8d8a74e6afad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:07.345631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:07.346345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d455f96a-b43d-4c88-8f36-3a1c08a5abe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:07.347901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1327/2638] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:07 2022] DELETE /identity/v3/projects/0392cedb0ff64712aba8d8a74e6afad6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:07.352733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.353216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.353312 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.373100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:07.374334 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:07.435011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1312/2639] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:07.442715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.443006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.443291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.464020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:07.465232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:07.473295 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1328/2640] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:07.477745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.490740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ea6c6-e13e-4522-989c-dc262a6f9c6c None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.496484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.497221 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.497509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.497782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.498511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:07.498818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.500418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.516115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1313/2641] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:07.517714 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.527453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f3f18-9be7-4aa2-adb9-38c2dbb9342b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.532242 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.532812 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.533089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.533374 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.534129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.536691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.543439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:07.573708 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1329/2642] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:07.575539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.585628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:07.587452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5b759-6698-439f-b879-97bee8320fb4 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.658124 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.658745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.659024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.659297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.659962 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.661904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.686375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1314/2643] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:07.692165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.703560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce441761-1a2b-4ece-8862-9fbb2f8f2156 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.708232 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.708861 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.709394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.709673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.710416 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.712061 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.722356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1330/2644] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:07.726456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.737320 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374ed27-4604-46ee-8dc6-1e1c85410710 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.743207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.743837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.744115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.744391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.745092 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.746743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23ead7-39b1-4b62-89ad-2e457eeb5b51 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.758604 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1315/2645] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:07.763180 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:07.774425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acedef-e52d-4dd8-a9eb-1698c899d2b7 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:07.779203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:07.779886 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.780164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.780437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] PATH_INFO: `/v3/projects/8ae84b557c2a45bbad97db92889bd3b3/users/b01349ac58be4a85aabba1bbc3fc3c78/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.803214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae84b557c2a45bbad97db92889bd3b3, user_id=b01349ac58be4a85aabba1bbc3fc3c78, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:07.806645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:07.821713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1331/2646] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:07 2022] PUT /identity/v3/projects/8ae84b557c2a45bbad97db92889bd3b3/users/b01349ac58be4a85aabba1bbc3fc3c78/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:07.832090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:07.832376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:07.832651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:07.862388 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] MFA Rules not processed for user `b01349ac58be4a85aabba1bbc3fc3c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:07.863622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:07.958035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1316/2647] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:07.963438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.160070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0503b-e946-4a30-ab84-779c467c05a3 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.165574 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.166239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.166522 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.166793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.167458 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.169377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.195907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1332/2648] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:07 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:08.200160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.211698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1351c1f-27dc-4200-8e2a-f21c730b93ee None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.270921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.271558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.271838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.272115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.272800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.275242 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.296638 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1317/2649] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:08 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:08.300617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.310830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51da69-6400-42d7-88f6-c2336d7dce8b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.315367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.315984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.316266 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.316544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.317237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.318877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.329741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1333/2650] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:08.333995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.340428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:08.346583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682da2-6abe-49fd-bb82-d3e7a87f769f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.352423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.353125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.353707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.353707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] PATH_INFO: `/v3/projects/5519c50e77c140139dfc23e3829eab8a/users/4aeae8bb204c48dba01ec6b88af339cc/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.368563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5519c50e77c140139dfc23e3829eab8a, user_id=4aeae8bb204c48dba01ec6b88af339cc, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.371275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.384659 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1318/2651] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:08 2022] PUT /identity/v3/projects/5519c50e77c140139dfc23e3829eab8a/users/4aeae8bb204c48dba01ec6b88af339cc/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:08.388915 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.399422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c973-3ff6-4a9a-b4e3-baad430b95e7 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.461742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.462377 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.462659 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.462957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.463650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.465225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a04ab-e6c7-4bc9-b137-c0ef2da0f50a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.475244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1334/2652] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:08.479442 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.490096 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258d9a7-3de4-4c7b-95d1-da141a59c856 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.494780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.495448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.495726 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.496002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] PATH_INFO: `/v3/projects/5519c50e77c140139dfc23e3829eab8a/users/4aeae8bb204c48dba01ec6b88af339cc/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.497971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5519c50e77c140139dfc23e3829eab8a, user_id=4aeae8bb204c48dba01ec6b88af339cc, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.500592 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.512735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1319/2653] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:08 2022] PUT /identity/v3/projects/5519c50e77c140139dfc23e3829eab8a/users/4aeae8bb204c48dba01ec6b88af339cc/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:08.524760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c060631-3a60-4f49-8012-5920be499fef None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.525046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c060631-3a60-4f49-8012-5920be499fef None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.525323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c060631-3a60-4f49-8012-5920be499fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.553569 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3c060631-3a60-4f49-8012-5920be499fef None None] MFA Rules not processed for user `4aeae8bb204c48dba01ec6b88af339cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:08.554352 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3c060631-3a60-4f49-8012-5920be499fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:08.564217 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:08.625244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1335/2654] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:08.637275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.639212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:08.702618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711ab90-9cf0-4891-a45c-cf32c726a83f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.706434 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.707070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.707354 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.707628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.717831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.719237 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.729349 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1320/2655] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:08.776521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c060631-3a60-4f49-8012-5920be499fef None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:08.778109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3c060631-3a60-4f49-8012-5920be499fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:08.790312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c060631-3a60-4f49-8012-5920be499fef None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:08.793975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:08.794597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:08.794874 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:08.795141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:08.852881 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:08.854357 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:08.863107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1336/2656] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:11.311623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:11.313656 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:11.417774 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b6d3-c086-4888-90e0-2989dcb5b9a0 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:11.423590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:11.424360 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:11.424878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:11.425052 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:11.426996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:11.440795 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:11.442529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171e7e3-66c6-43db-8130-1dec44466bcd service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:11.455873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1321/2657] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:12.250420 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.323929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff24a6d-0f2e-4609-b1a2-44dbac281a5e service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.328403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.329049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.329326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.329597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] PATH_INFO: `/v3/users/b01349ac58be4a85aabba1bbc3fc3c78` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.330784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01349ac58be4a85aabba1bbc3fc3c78)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.332698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.378683 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01349ac58be4a85aabba1bbc3fc3c78'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.379824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:12.394225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01349ac58be4a85aabba1bbc3fc3c78'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.400847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01349ac58be4a85aabba1bbc3fc3c78'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.401561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01349ac58be4a85aabba1bbc3fc3c78'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.402179 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01349ac58be4a85aabba1bbc3fc3c78'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.414769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1337/2658] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:12 2022] DELETE /identity/v3/users/b01349ac58be4a85aabba1bbc3fc3c78 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:12.419351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.419879 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.419995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.439878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:12.441128 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.489153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:12.502593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1322/2659] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:12.512182 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.513439 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.579184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc21933-e093-4cf0-956a-a146310a7095 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.582930 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.583664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.583925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.584742 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.594843 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.596275 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c8abb-4f34-4d45-9b9e-32baa389db51 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.606732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1338/2660] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:12.654133 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.655753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.669393 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747f3f6-981c-46c9-b4ae-339fccdcf558 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.674178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.674811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.675090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.675361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] PATH_INFO: `/v3/users/44027884ef6a4175845ceb7bc05467f8` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.676581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=44027884ef6a4175845ceb7bc05467f8)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.678577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.729058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44027884ef6a4175845ceb7bc05467f8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.729876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44027884ef6a4175845ceb7bc05467f8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.741363 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44027884ef6a4175845ceb7bc05467f8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.750615 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44027884ef6a4175845ceb7bc05467f8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.760330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44027884ef6a4175845ceb7bc05467f8'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.767182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1323/2661] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:12 2022] DELETE /identity/v3/users/44027884ef6a4175845ceb7bc05467f8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:12.771986 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.772273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.772550 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.792542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:12.793741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.855427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1339/2662] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:12.865769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.868033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.884885 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:12.886361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:12.901028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b54f-bf31-42b6-b3cd-2976bb2c67bf None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.906633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:12.907454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:12.907737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:12.908173 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] PATH_INFO: `/v3/projects/8ae84b557c2a45bbad97db92889bd3b3` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:12.909349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae84b557c2a45bbad97db92889bd3b3)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:12.911685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:12.952663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da678-c24b-40f9-bac0-8055a842f4dc admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:12.962159 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae84b557c2a45bbad97db92889bd3b3', 'request_id': 'req-14c958c4-e832-49fa-b12e-d3da6a6cea27'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.970572 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae84b557c2a45bbad97db92889bd3b3', 'request_id': 'req-14c958c4-e832-49fa-b12e-d3da6a6cea27'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.977802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae84b557c2a45bbad97db92889bd3b3', 'request_id': 'req-14c958c4-e832-49fa-b12e-d3da6a6cea27'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.980040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:12.990440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] The token cache is being invalidate because project 8ae84b557c2a45bbad97db92889bd3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:12.990763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.991844 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:12.994316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1340/2663] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:12 2022] DELETE /identity/v3/projects/8ae84b557c2a45bbad97db92889bd3b3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:12.998048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:13.000967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.009338 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.010002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.010277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.010548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.012401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.047917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:13.068653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958c4-e832-49fa-b12e-d3da6a6cea27 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:13.076012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.076782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.077080 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.077361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] PATH_INFO: `/v3/users/4aeae8bb204c48dba01ec6b88af339cc` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.078726 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aeae8bb204c48dba01ec6b88af339cc)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.081207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.091254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.092706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.104186 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1324/2664] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:13.139484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aeae8bb204c48dba01ec6b88af339cc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.154472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aeae8bb204c48dba01ec6b88af339cc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.161749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aeae8bb204c48dba01ec6b88af339cc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.162529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aeae8bb204c48dba01ec6b88af339cc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.163155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aeae8bb204c48dba01ec6b88af339cc'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.177164 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1341/2665] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:12 2022] DELETE /identity/v3/users/4aeae8bb204c48dba01ec6b88af339cc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:13.502650 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:13.503203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:13.505552 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.516021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003060fa-06a4-42a6-88ca-7cfffacd63df service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:13.598515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.599178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.599459 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.599734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] PATH_INFO: `/v3/projects/5519c50e77c140139dfc23e3829eab8a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.600927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5519c50e77c140139dfc23e3829eab8a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.602956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.631216 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb6765-a2c5-4cde-9efd-27feb7778257 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:13.636530 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.637280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.637598 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.637869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] PATH_INFO: `/v3/projects/6bb18b2d219347f99638d5f93ea4e2cd` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.639263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb18b2d219347f99638d5f93ea4e2cd)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.641165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.655231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:13.666933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5519c50e77c140139dfc23e3829eab8a', 'request_id': 'req-3d7ad783-c980-4341-bbad-adb26ed3470b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.677627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5519c50e77c140139dfc23e3829eab8a', 'request_id': 'req-3d7ad783-c980-4341-bbad-adb26ed3470b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.684318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5519c50e77c140139dfc23e3829eab8a', 'request_id': 'req-3d7ad783-c980-4341-bbad-adb26ed3470b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.698677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] The token cache is being invalidate because project 5519c50e77c140139dfc23e3829eab8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:13.699224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.700036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-3d7ad783-c980-4341-bbad-adb26ed3470b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.700755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb18b2d219347f99638d5f93ea4e2cd', 'request_id': 'req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.701802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1325/2666] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:13 2022] DELETE /identity/v3/projects/5519c50e77c140139dfc23e3829eab8a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:13.706909 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.707165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.707625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.716109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb18b2d219347f99638d5f93ea4e2cd', 'request_id': 'req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.723627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb18b2d219347f99638d5f93ea4e2cd', 'request_id': 'req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.730299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:13.731525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.736610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] The token cache is being invalidate because project 6bb18b2d219347f99638d5f93ea4e2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:13.736899 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.737931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.739690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1342/2667] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:13 2022] DELETE /identity/v3/projects/6bb18b2d219347f99638d5f93ea4e2cd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:13.743553 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:13.744994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.811358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1326/2668] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:13.812212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:13.813870 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8ca11-33db-4b42-a9fb-2f00ada38b2a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:13.815123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.820085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:13.822523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.823225 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.823516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.823771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] PATH_INFO: `/v3/users/51cbbef751324dcaa235c5f0a35b876b` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.824956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=51cbbef751324dcaa235c5f0a35b876b)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.826939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.876677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b8469-2a31-4910-b068-1ef65154d10f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:13.878046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51cbbef751324dcaa235c5f0a35b876b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.880569 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:13.881199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.881486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.881759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.884101 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.890940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51cbbef751324dcaa235c5f0a35b876b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.897857 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51cbbef751324dcaa235c5f0a35b876b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.899178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51cbbef751324dcaa235c5f0a35b876b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.899808 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ecd29f36-72b1-4228-b766-db160b6885ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51cbbef751324dcaa235c5f0a35b876b'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:13.913393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1343/2669] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:13 2022] DELETE /identity/v3/users/51cbbef751324dcaa235c5f0a35b876b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:13.914691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:13.914979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:13.915258 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:13.940022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:13.942200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:13.958287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:13.961441 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:13.963013 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:13.978212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1327/2670] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:14.012591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1344/2671] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:14.016872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.018341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.033716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4199a4-95ed-4908-9553-950e5da76cc1 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.042484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.043204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.043622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.043990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.045187 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:14.045625 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.047775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.062732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1328/2672] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:14.067698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.079002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683cc27-9356-4914-a02e-e13bce2b3b6c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.083831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.084727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.085008 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.085337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.086070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.087988 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.118512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1345/2673] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:14.123070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.135817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cbb2c-930b-4d36-8ab8-c9ef99349ac7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.196396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.197211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.197500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.197782 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.198551 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.200474 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.225740 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1329/2674] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/users => generated 305 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:14.227036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.239069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05fc1-68b8-4a87-a1f8-1cd32ac71d5a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.249556 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.250293 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.250570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.250838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.251616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.253251 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.273430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1346/2675] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:14.274407 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.286558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1569d-2f16-4649-b80f-5252dccd3605 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.293489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.294175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.294458 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.296078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.296078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.297627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2f224-ab57-406a-a56f-d082ca860440 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.309758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1330/2676] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:14.314063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.324563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffd425-33df-4e95-8d15-e5f317917c97 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.329762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.330408 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.330718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.330994 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] PATH_INFO: `/v3/projects/a14ca7a9688b4ad1b2d97714ef6da8cf/users/4326696cedc042b584cbd60af91b7e3a/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.348417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14ca7a9688b4ad1b2d97714ef6da8cf, user_id=4326696cedc042b584cbd60af91b7e3a, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.352775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.368852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1347/2677] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:14 2022] PUT /identity/v3/projects/a14ca7a9688b4ad1b2d97714ef6da8cf/users/4326696cedc042b584cbd60af91b7e3a/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:14.381881 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.382176 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.382454 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.392100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.393806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.415371 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] MFA Rules not processed for user `4326696cedc042b584cbd60af91b7e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:14.416730 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.423246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:14.493987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92856b37-2194-4521-bd75-7f5e228d3746 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.496760 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1331/2678] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:14.500858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.500858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.500858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.500858 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] PATH_INFO: `/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.501286 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b10d3f783c44ddbebcf3828ae3322c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.502333 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.502933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.514322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ae50-cb4e-4eee-b202-bc8b8e268e51 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.520072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.521908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.522139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.522410 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.523088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.525083 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.557292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b10d3f783c44ddbebcf3828ae3322c', 'request_id': 'req-2b6e9d4f-e84a-42c3-a249-0473607aa605'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:14.559028 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1332/2679] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:14.564736 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.569330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b10d3f783c44ddbebcf3828ae3322c', 'request_id': 'req-2b6e9d4f-e84a-42c3-a249-0473607aa605'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:14.577661 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b10d3f783c44ddbebcf3828ae3322c', 'request_id': 'req-2b6e9d4f-e84a-42c3-a249-0473607aa605'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:14.582140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:14.594130 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] The token cache is being invalidate because project a2b10d3f783c44ddbebcf3828ae3322c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:14.595220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:14.596395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2b6e9d4f-e84a-42c3-a249-0473607aa605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:14.598725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1348/2680] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:14 2022] DELETE /identity/v3/projects/a2b10d3f783c44ddbebcf3828ae3322c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:14.603957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f4f9-a302-47ef-8663-512080cb064e None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.604394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f4f9-a302-47ef-8663-512080cb064e None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.604674 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f4f9-a302-47ef-8663-512080cb064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.631238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-1724f4f9-a302-47ef-8663-512080cb064e None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:14.632588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1724f4f9-a302-47ef-8663-512080cb064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.637750 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ea44-ed08-4f40-b9cd-49612fe301fb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.642784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.643486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.643840 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.644191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.644924 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.646994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.653872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1349/2681] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:14.655555 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.655848 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.656128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.674267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1333/2682] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/users => generated 310 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:14.678592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:14.678780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.680425 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.681130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:14.688736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1350/2683] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:14.694170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.698069 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdd95b-0640-4a54-bb62-857df9427920 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.703670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.704311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.704596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.704873 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.705580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.706521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed9654-6a6d-4031-8568-4228cc6df8e1 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.707236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.712120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.712748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.713026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.713297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.714064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:14.714372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.716038 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.720064 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1334/2684] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:14.724111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.734270 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1351/2685] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:14.736262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.738738 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd7bad-8f99-4186-b31e-00350f6436d7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.746100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.746637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e5114-2235-42a9-9877-81b7092455ca admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.747004 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.747353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.747696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] PATH_INFO: `/v3/projects/bf76bb91b786454f86f54b26027f5527/users/774d7b38252a4de8b53c2c8a665e699a/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.753561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.755126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.755126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.755126 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.755813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.758512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.764166 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf76bb91b786454f86f54b26027f5527, user_id=774d7b38252a4de8b53c2c8a665e699a, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.766860 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.782837 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1335/2686] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:14 2022] PUT /identity/v3/projects/bf76bb91b786454f86f54b26027f5527/users/774d7b38252a4de8b53c2c8a665e699a/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:14.787081 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.792786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1352/2687] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:14.796931 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.838916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:14.863524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9d7e-e8af-4d65-8697-447e9a58e097 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.868263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9a687-58e6-438b-a026-2101d7faa8bc admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.869466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.870147 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.870430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.870712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.871424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.873344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.874426 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.875238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.875669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.876123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.876948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.879082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.885305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1336/2688] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:14.890067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.902213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d853670-078f-40cf-94f1-7d931a1e86e7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.907228 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1353/2689] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:14.907412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:14.908141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.908418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.908694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] PATH_INFO: `/v3/projects/bf76bb91b786454f86f54b26027f5527/users/774d7b38252a4de8b53c2c8a665e699a/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.909150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:14.910856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf76bb91b786454f86f54b26027f5527, user_id=774d7b38252a4de8b53c2c8a665e699a, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:14.913559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87a26-f10d-431e-89f1-8825b0add446 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:14.926868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d579740-2bcb-400a-aa66-4e1ef7cf8081 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:14.927120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1337/2690] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:14 2022] PUT /identity/v3/projects/bf76bb91b786454f86f54b26027f5527/users/774d7b38252a4de8b53c2c8a665e699a/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:14.937210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:14.937528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:14.937868 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:14.970899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] MFA Rules not processed for user `774d7b38252a4de8b53c2c8a665e699a`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:14.972154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.000706 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.001356 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.001641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.001950 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.002835 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.004432 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.020787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1354/2691] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:15.021524 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.032517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225578b-44e9-4b96-baa6-88f39b91435e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.037525 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.038380 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.038775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.039194 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.040112 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.041827 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.052453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1338/2692] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:15.061208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1355/2693] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:15.062335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.064262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.066815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.077307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34939a4f-7c97-4a47-9b08-4e6839f7c8b9 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.083255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.084039 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.084234 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.084784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] PATH_INFO: `/v3/projects/9c294b1151ab4da38a2e09535f5bdeb4/users/d8434ce0dd2646ab8c4f787c2adc0da2/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.096864 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.104914 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c294b1151ab4da38a2e09535f5bdeb4, user_id=d8434ce0dd2646ab8c4f787c2adc0da2, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.108266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa7169-5dfc-4049-9222-fc920f1c313b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.124619 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1339/2694] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:15 2022] PUT /identity/v3/projects/9c294b1151ab4da38a2e09535f5bdeb4/users/d8434ce0dd2646ab8c4f787c2adc0da2/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:15.136819 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.137115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.137413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.142299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe5e5-4c44-4de7-9b03-29fe672f6cb5 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.168408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] MFA Rules not processed for user `d8434ce0dd2646ab8c4f787c2adc0da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:15.169775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.202128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.202757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.203035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.203303 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.245119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1340/2695] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:15.249709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.265776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.272607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.274136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.284508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1356/2696] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:15.337139 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15929-1356-4b5f-9114-1bf484fb26d6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.343516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.344148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.344430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.344711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.345375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.347525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.379228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1341/2697] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:15 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:15.383513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.395658 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e81-5309-46ed-9a2a-529cbe3e00e8 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.464822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.465488 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.465764 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.466064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.466767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.468738 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.472462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.493200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1357/2698] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:15 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:15.497047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.510322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdce2-af18-4c3d-8a23-e1de87247c06 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.517651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.518312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.518594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.518871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.520122 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.522602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.536353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1358/2699] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:15.538346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618c5f8-41f6-4614-a545-72599caf7503 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.540953 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.543321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.543943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.544265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.544540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.546588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.548532 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.552428 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a159b-3972-4658-aa41-a19b8d73ad15 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.559020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.561743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.561743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.566354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] PATH_INFO: `/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb/users/224ddf1f60d74eda8d0b88ad0144e20f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.588041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba29166b33ab4ecab6ffe5dba29523cb, user_id=224ddf1f60d74eda8d0b88ad0144e20f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.591310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.607830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1359/2700] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:15 2022] PUT /identity/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb/users/224ddf1f60d74eda8d0b88ad0144e20f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:15.611875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.624633 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.626564 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.638460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1342/2701] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:15.685570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328789c6-06c9-426b-b4de-c34c86eb0407 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.690253 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.690869 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.691149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.691422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.692127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.694396 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236172a-cf81-448b-8482-19070d5ea173 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.709884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1360/2702] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:15.712171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.723335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be999bb-7f66-45ee-85d5-68a5751fb2b5 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.729267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.730220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.730686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.731170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] PATH_INFO: `/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb/users/224ddf1f60d74eda8d0b88ad0144e20f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.733607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba29166b33ab4ecab6ffe5dba29523cb, user_id=224ddf1f60d74eda8d0b88ad0144e20f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:15.737480 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:15.749286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1343/2703] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:15 2022] PUT /identity/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb/users/224ddf1f60d74eda8d0b88ad0144e20f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:15.759609 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.759901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.760165 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:15.793058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] MFA Rules not processed for user `224ddf1f60d74eda8d0b88ad0144e20f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:15.794306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.879249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1361/2704] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:15.891471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:15.893222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:15.965534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:15.983079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4384-9f5c-49d1-a61a-92209d822fb2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:15.989328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:15.990058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:15.990403 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:15.990746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:16.002145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:16.004244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:16.019366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1344/2705] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:16.101340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:16.118220 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f815-55cb-4f5a-86e4-d8ce20ec2e0c None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:16.123956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:16.125074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:16.125394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:16.125743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:16.170944 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:16.173027 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:16.205537 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:16.207047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:16.213114 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:16.221024 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1362/2706] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:16.269265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed808-3735-4cf5-8729-1f38272b208c service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:16.276162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:16.276971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:16.277503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:16.278010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:16.280401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:16.295921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:16.297877 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66644-e409-449c-b908-e31fe78de289 service nova] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:16.310939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1345/2707] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:26.304427 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=42 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:27.253328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.315030 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3841c6-6266-4858-b4e1-2bc2f2854fcf service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:27.319383 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:27.320016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.320379 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.320695 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] PATH_INFO: `/v3/users/4326696cedc042b584cbd60af91b7e3a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.321945 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4326696cedc042b584cbd60af91b7e3a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:27.324094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:27.372531 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4326696cedc042b584cbd60af91b7e3a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.384564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4326696cedc042b584cbd60af91b7e3a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.391823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4326696cedc042b584cbd60af91b7e3a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.392635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4326696cedc042b584cbd60af91b7e3a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.393394 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4326696cedc042b584cbd60af91b7e3a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.407974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1363/2708] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:27 2022] DELETE /identity/v3/users/4326696cedc042b584cbd60af91b7e3a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:27.412433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.412939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.413053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.440541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:27.442046 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:27.504462 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1346/2709] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:27.514551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.516254 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:27.575407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6aa52-eb86-42e4-90a8-df6a09ac1938 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:27.579006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:27.579629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.579912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.580187 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.590120 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:27.591560 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:27.601151 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1364/2710] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:27.874728 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.876213 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:27.888793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc03d6-de9a-494d-a17a-558ee658cdc2 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:27.893553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:27.894305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:27.894586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:27.894861 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] PATH_INFO: `/v3/projects/a14ca7a9688b4ad1b2d97714ef6da8cf` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:27.896037 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14ca7a9688b4ad1b2d97714ef6da8cf)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:27.897986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:27.946222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14ca7a9688b4ad1b2d97714ef6da8cf', 'request_id': 'req-a7389e66-e40e-4afc-b9c2-cbb990ea2175'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.953034 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14ca7a9688b4ad1b2d97714ef6da8cf', 'request_id': 'req-a7389e66-e40e-4afc-b9c2-cbb990ea2175'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.959264 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14ca7a9688b4ad1b2d97714ef6da8cf', 'request_id': 'req-a7389e66-e40e-4afc-b9c2-cbb990ea2175'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.970177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] The token cache is being invalidate because project a14ca7a9688b4ad1b2d97714ef6da8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:27.970466 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.971146 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:27.972645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1347/2711] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:27 2022] DELETE /identity/v3/projects/a14ca7a9688b4ad1b2d97714ef6da8cf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:27.976713 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:27.986745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cf2b7-4d3d-4a7f-b2b5-4e9a97ce0a5e service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.007072 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:28.047183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.047816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.048094 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.048367 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] PATH_INFO: `/v3/users/774d7b38252a4de8b53c2c8a665e699a` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.049606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774d7b38252a4de8b53c2c8a665e699a)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.051623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.096566 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774d7b38252a4de8b53c2c8a665e699a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.108721 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774d7b38252a4de8b53c2c8a665e699a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.115495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774d7b38252a4de8b53c2c8a665e699a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.116154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774d7b38252a4de8b53c2c8a665e699a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.116757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-20d7224f-99a9-441c-beeb-830e032b8ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774d7b38252a4de8b53c2c8a665e699a'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.130302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1365/2712] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:27 2022] DELETE /identity/v3/users/774d7b38252a4de8b53c2c8a665e699a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:28.403907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.405632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:28.422292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7389e66-e40e-4afc-b9c2-cbb990ea2175 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.486764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.487429 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.487929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.488322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] PATH_INFO: `/v3/projects/bf76bb91b786454f86f54b26027f5527` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.489780 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf76bb91b786454f86f54b26027f5527)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.491469 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.526344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:28.545008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf76bb91b786454f86f54b26027f5527', 'request_id': 'req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.552259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf76bb91b786454f86f54b26027f5527', 'request_id': 'req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.558368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf76bb91b786454f86f54b26027f5527', 'request_id': 'req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.568949 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] The token cache is being invalidate because project bf76bb91b786454f86f54b26027f5527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:28.569232 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.569899 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ddaf10e8-1f1b-4bb5-8c3a-28c2024d8761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:28.571414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1348/2713] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:28 2022] DELETE /identity/v3/projects/bf76bb91b786454f86f54b26027f5527 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:28.576819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.577281 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.577391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.597411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:28.598663 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:28.656860 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1366/2714] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:28.665289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.665599 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.665904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.685623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:28.686867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:28.695130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1349/2715] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:28.700718 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.711929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd384-6754-49b0-b913-d6823b7c1b48 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.717606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.718249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.718526 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.718796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.719727 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:28.720084 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.721876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.734529 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1367/2716] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:28.738907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.748225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12762d-da84-4ae0-b066-f559f1e166fc None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.753123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.753823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.754148 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.754492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.755273 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.757292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.785274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1350/2717] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:28 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:28.789633 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.799743 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bbeb0-205b-443b-93ee-db16e17cf2cb admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.828650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:28.862692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.863305 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.863579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.863847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.864505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.866517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.889212 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1368/2718] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:28 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:28.893451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.903637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38992-37d3-4383-8716-83e4ae4137f3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.908353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.908906 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.909224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.909483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.910211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.911814 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.921694 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1351/2719] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:28.925631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.936450 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3afb2-ddd8-470e-94bc-478fa781b992 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.940998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.941618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.941926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.942288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.943013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.944579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0def8-4ccb-40cf-9fff-e1275b22058f admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:28.955533 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1369/2720] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:28.960063 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:28.970018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c2d52-9bd7-4669-bee0-311c64cb2548 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:28.974608 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:28.975259 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:28.975544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:28.975943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] PATH_INFO: `/v3/projects/9a518cf04eaf44d9a40277d71fb29008/users/5d0121f65d624bbf898582fb820fe465/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:28.990143 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a518cf04eaf44d9a40277d71fb29008, user_id=5d0121f65d624bbf898582fb820fe465, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:28.992775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.006912 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1352/2721] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:28 2022] PUT /identity/v3/projects/9a518cf04eaf44d9a40277d71fb29008/users/5d0121f65d624bbf898582fb820fe465/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:29.073400 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.073692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.074018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.101772 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] MFA Rules not processed for user `5d0121f65d624bbf898582fb820fe465`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:29.103016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.165103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1370/2722] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:29.169699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.234318 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76ad5-fa69-42b1-a969-649089f6ff9c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.239029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.239655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.239932 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.240202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.240857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.242792 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.272456 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1353/2723] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:29 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:29.276673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.287237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc0d41-7598-480d-8199-144e24cd009b None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.341952 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.342570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.342847 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.343117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.343771 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.345669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.376896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1371/2724] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:29 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:29.378919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.391056 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1c19-d0b5-4240-95ec-b2088bba2605 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.395991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.396586 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.396863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.397136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.397827 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.399451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.411468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1354/2725] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:29.416033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.426519 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90c0c-a626-4d99-a3d9-caf537830587 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.431777 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.433052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.433330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.433603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] PATH_INFO: `/v3/projects/1c4b06d5bafd40889320955d03049505/users/adcab6472b4c4ec0a6c27a1ae4179f01/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.448936 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4b06d5bafd40889320955d03049505, user_id=adcab6472b4c4ec0a6c27a1ae4179f01, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.451616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.464585 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1372/2726] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:29 2022] PUT /identity/v3/projects/1c4b06d5bafd40889320955d03049505/users/adcab6472b4c4ec0a6c27a1ae4179f01/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:29.468680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.478806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23532aac-2eab-4ab9-8ede-b1bf13fe4de4 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.499585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:29.538227 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.538857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.539136 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.539412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.540211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.542372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c3c1e-14b8-4e7f-97f5-e02b95571811 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.556049 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1355/2727] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:29.557782 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.563405 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:29.572372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034772-17d9-477f-8f16-ac3a97c85d1e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.577954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.578954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.578954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.579256 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] PATH_INFO: `/v3/projects/1c4b06d5bafd40889320955d03049505/users/adcab6472b4c4ec0a6c27a1ae4179f01/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.584093 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4b06d5bafd40889320955d03049505, user_id=adcab6472b4c4ec0a6c27a1ae4179f01, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.585447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.596903 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1373/2728] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:29 2022] PUT /identity/v3/projects/1c4b06d5bafd40889320955d03049505/users/adcab6472b4c4ec0a6c27a1ae4179f01/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:29.607971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.608329 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.608970 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.638399 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] MFA Rules not processed for user `adcab6472b4c4ec0a6c27a1ae4179f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:29.639708 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.729917 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.732387 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.733201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1356/2729] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:29.744385 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.745779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.802686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6184ef9-f145-4e00-bcf9-3363084de5d7 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.803184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6599cf-b347-43ae-ae37-45541aaa0dae admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.806409 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.807033 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.807344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.807626 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.807832 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.808467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.808747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.809016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] PATH_INFO: `/v3/users/d8434ce0dd2646ab8c4f787c2adc0da2` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.810210 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8434ce0dd2646ab8c4f787c2adc0da2)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.812115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.816878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:29.818356 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a37e4e-ecaf-4743-b652-9c67f70d2cb6 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:29.828041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1357/2730] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:29.863040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8434ce0dd2646ab8c4f787c2adc0da2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:29.875897 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8434ce0dd2646ab8c4f787c2adc0da2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:29.883323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8434ce0dd2646ab8c4f787c2adc0da2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:29.884064 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8434ce0dd2646ab8c4f787c2adc0da2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:29.884707 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8434ce0dd2646ab8c4f787c2adc0da2'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:29.899175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1374/2731] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:29 2022] DELETE /identity/v3/users/d8434ce0dd2646ab8c4f787c2adc0da2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:29.904618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.904900 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.905174 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.915642 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:29.917813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.925252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:29.926473 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.973542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:29.988576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157b8d0-7f08-4905-888e-f83990f27965 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:29.989540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1358/2732] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:29.992489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:29.993544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:29.993544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:29.993826 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:29.995660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:29.999647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:30.000974 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:30.018069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:30.057196 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:30.058514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea086-e600-4cc9-a0e0-4f157e155f8f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:30.058672 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:30.062132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:30.062745 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.063018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.063289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.068021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1375/2733] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:30.073220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:30.074654 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:30.085129 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1359/2734] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:30.367181 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:30.370987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:30.387750 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8819-3cd0-4523-bd05-290a90752983 service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:30.392616 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:30.393475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.394043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.394262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] PATH_INFO: `/v3/projects/9c294b1151ab4da38a2e09535f5bdeb4` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.395656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c294b1151ab4da38a2e09535f5bdeb4)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:30.398154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:30.447512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c294b1151ab4da38a2e09535f5bdeb4', 'request_id': 'req-ad72d752-f77c-4138-8a4f-5e39b0413335'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.454787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c294b1151ab4da38a2e09535f5bdeb4', 'request_id': 'req-ad72d752-f77c-4138-8a4f-5e39b0413335'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.461201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c294b1151ab4da38a2e09535f5bdeb4', 'request_id': 'req-ad72d752-f77c-4138-8a4f-5e39b0413335'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.471976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] The token cache is being invalidate because project 9c294b1151ab4da38a2e09535f5bdeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:30.472297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.472951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.474501 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1376/2735] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:30 2022] DELETE /identity/v3/projects/9c294b1151ab4da38a2e09535f5bdeb4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:30.478559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:30.488165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5393d-1a19-4a75-a82a-3b4adf750e79 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:30.544776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:30.545412 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:30.545693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:30.545995 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] PATH_INFO: `/v3/users/224ddf1f60d74eda8d0b88ad0144e20f` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:30.547215 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224ddf1f60d74eda8d0b88ad0144e20f)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:30.549299 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:30.597022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224ddf1f60d74eda8d0b88ad0144e20f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.597663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224ddf1f60d74eda8d0b88ad0144e20f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.609380 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224ddf1f60d74eda8d0b88ad0144e20f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.616649 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224ddf1f60d74eda8d0b88ad0144e20f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.624245 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f06a923a-904a-4cae-b56f-e87e00e1f678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224ddf1f60d74eda8d0b88ad0144e20f'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:30.630110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1360/2736] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:30 2022] DELETE /identity/v3/users/224ddf1f60d74eda8d0b88ad0144e20f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:30.997480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:30.999103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:31.012752 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d752-f77c-4138-8a4f-5e39b0413335 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.071939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.072580 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.072896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.073172 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] PATH_INFO: `/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.074347 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba29166b33ab4ecab6ffe5dba29523cb)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.076240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.121109 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:31.122369 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba29166b33ab4ecab6ffe5dba29523cb', 'request_id': 'req-f1597a6d-4c19-4f8d-9865-8da175575cc9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:31.130104 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba29166b33ab4ecab6ffe5dba29523cb', 'request_id': 'req-f1597a6d-4c19-4f8d-9865-8da175575cc9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:31.136435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba29166b33ab4ecab6ffe5dba29523cb', 'request_id': 'req-f1597a6d-4c19-4f8d-9865-8da175575cc9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:31.146702 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] The token cache is being invalidate because project ba29166b33ab4ecab6ffe5dba29523cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:31.146991 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:31.147638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-f1597a6d-4c19-4f8d-9865-8da175575cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:31.149149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1377/2737] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:30 2022] DELETE /identity/v3/projects/ba29166b33ab4ecab6ffe5dba29523cb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:31.155007 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.155512 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.155817 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.177019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:31.178292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:31.183921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:31.239580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1361/2738] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:31.247386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.247665 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.247943 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.267353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:31.268565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:31.277343 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1378/2739] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:31.281602 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.292260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1bf1e-7cbe-4c57-9b8f-c0678017eda6 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.296875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.297488 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.297775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.298090 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.298804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:31.299100 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.300709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.312548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1362/2740] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:31.316732 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.326135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84816-693b-42f8-8f95-439489aa4d8f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.331157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.331766 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.332043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.332315 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.332971 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.335262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.367697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1379/2741] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:31.373330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.389293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9774-4be1-4460-8929-93d35ed79d0c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.457171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.457988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.458272 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.458542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.459211 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.461620 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.485257 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1363/2742] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:31.489361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.500511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561820a0-fc41-4313-8fdd-33f6206ca373 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.505736 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.506473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.506828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.507195 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.507975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.509615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.521767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1380/2743] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 09:23:31.526460 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.537387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08a6f6-9583-4803-9829-d01ba645dcf6 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.543060 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.543670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.543945 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.544216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.544907 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.546534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4e695-2625-4c48-b116-5dd347de9722 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.556916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1364/2744] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:31.561697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.572040 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108dbd5-cf3a-479e-8916-136384009b77 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.577565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.578372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.578650 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.578964 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] PATH_INFO: `/v3/projects/ba2421e2ca164930bfff36c222b85a44/users/95c8c333ddde47448db5fdff3398f792/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.595841 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2421e2ca164930bfff36c222b85a44, user_id=95c8c333ddde47448db5fdff3398f792, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.598544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.614013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1381/2745] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:31 2022] PUT /identity/v3/projects/ba2421e2ca164930bfff36c222b85a44/users/95c8c333ddde47448db5fdff3398f792/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:31.630359 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.630678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.630957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.660372 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] MFA Rules not processed for user `95c8c333ddde47448db5fdff3398f792`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:31.661719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:31.738515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1365/2746] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:31.743584 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.810512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699f928-73aa-462e-8871-494d03d9fd53 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.817668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.818325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.818606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.818883 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.819558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.821462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.851571 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1382/2747] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:31.856286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.867105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac101fd-b914-4d88-bdb6-416eb3360989 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.871594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:31.936519 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.937140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.937419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.937696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.938400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.940342 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.962391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1366/2748] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:31 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:31.966880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:31.980630 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8e800-e378-4945-bcd4-9b26b3d39bed admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:31.985393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:31.986055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:31.986341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:31.986617 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:31.987319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:31.988922 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:31.999681 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1383/2749] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:32.003829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.013823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8729035-688a-4947-b3f7-2365fd16a75c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.018568 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.019122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.019405 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.019678 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.020376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.022000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6c76a-11c8-4cc2-a677-51463b45f897 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.033310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1367/2750] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:32.037664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.048615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049eaf-f3ca-420d-b991-6012d5ab0360 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.053653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.054346 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.054621 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.054891 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] PATH_INFO: `/v3/projects/40ce0fc6215a4cbd917729eb16cc887c/users/f7f83edc430c45549699081a81e8969a/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.069824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ce0fc6215a4cbd917729eb16cc887c, user_id=f7f83edc430c45549699081a81e8969a, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.072486 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.085995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1384/2751] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:32 2022] PUT /identity/v3/projects/40ce0fc6215a4cbd917729eb16cc887c/users/f7f83edc430c45549699081a81e8969a/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:32.096676 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.096964 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.097244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.125800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] MFA Rules not processed for user `f7f83edc430c45549699081a81e8969a`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:32.127074 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:32.192419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1368/2752] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:32.196801 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.221992 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:32.259944 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0773b85-e2eb-480c-acd0-297c3b14ddcd admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.265407 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.266267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.266512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.266787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.267468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.269424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.296342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1385/2753] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:32.300381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.310984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a6097-72f4-47c0-ba06-2c9923d1de60 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.366152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.366765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.367051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.367330 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.367993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.369986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.393018 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1369/2754] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:32.396999 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.407737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe9e59-5b8a-4d1f-b884-b78a28788f87 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.412285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.412893 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.413168 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.413440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.414161 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.415851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.426386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1386/2755] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:32.430422 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.436831 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:32.440763 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88342b78-7adf-4e5e-a3b6-a424ea488aa1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.447029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.448216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.448499 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.448772 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] PATH_INFO: `/v3/projects/c79fc7dca8514cc5bc56bd60797978f5/users/f20bfb037437462b9c30b0ea39ea0de9/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.464190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79fc7dca8514cc5bc56bd60797978f5, user_id=f20bfb037437462b9c30b0ea39ea0de9, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.467141 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.480800 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1370/2756] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:32 2022] PUT /identity/v3/projects/c79fc7dca8514cc5bc56bd60797978f5/users/f20bfb037437462b9c30b0ea39ea0de9/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:32.485033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.494912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273d29b-2114-4469-bd47-4b272e7f2b61 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.550865 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.551489 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.551769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.552042 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.552729 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.554355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f37557-af45-465b-9dc6-53822a5077f4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.563890 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1387/2757] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:32.568243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.579123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812967ef-171a-4220-9411-5eb2dd14654c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.583988 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.584903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.585184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.585455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] PATH_INFO: `/v3/projects/c79fc7dca8514cc5bc56bd60797978f5/users/f20bfb037437462b9c30b0ea39ea0de9/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.587452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79fc7dca8514cc5bc56bd60797978f5, user_id=f20bfb037437462b9c30b0ea39ea0de9, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.590737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.600836 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1371/2758] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:32 2022] PUT /identity/v3/projects/c79fc7dca8514cc5bc56bd60797978f5/users/f20bfb037437462b9c30b0ea39ea0de9/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:32.667607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.668036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.668319 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.697469 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] MFA Rules not processed for user `f20bfb037437462b9c30b0ea39ea0de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:32.699753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:32.774517 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1388/2759] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:32.786952 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.787660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:32.858173 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c9a6-80b4-4e12-a6be-5fa4875d65ee admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.861818 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.862471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.862798 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.863024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.873452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:32.875152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:32.886260 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1372/2760] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:32.927418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:32.943089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76bfb-4287-4165-aa37-bbc1d866ede4 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:32.946866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:32.947493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:32.948345 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:32.948627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:32.962312 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:33.009125 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:33.011223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ee302-bebb-4f11-a07e-99529c07c5c5 service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:33.021007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1389/2761] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:34.874222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:34.875804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:34.941058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7cc21-f0e0-4e77-b35c-44fbfcc50008 service cinder] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:34.945581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:34.946267 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:34.946544 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:34.946826 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] PATH_INFO: `/v3/users/5d0121f65d624bbf898582fb820fe465` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:34.948023 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0121f65d624bbf898582fb820fe465)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:34.949978 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:34.995667 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0121f65d624bbf898582fb820fe465'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:34.996322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0121f65d624bbf898582fb820fe465'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.007829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0121f65d624bbf898582fb820fe465'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.014534 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0121f65d624bbf898582fb820fe465'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.015720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:35.024545 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0121f65d624bbf898582fb820fe465'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.031035 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1373/2762] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:34 2022] DELETE /identity/v3/users/5d0121f65d624bbf898582fb820fe465 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:35.035916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.036389 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.036487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.056872 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:35.058255 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:35.120429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1390/2763] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:35.130477 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.132108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:35.191191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daf8d-b4d7-4f93-9bde-e5404cf6bb97 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:35.195059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:35.195689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.195971 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.196247 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.205439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:35.206921 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:35.217302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1374/2764] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:35.480749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.482332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:35.496355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b46f1e-0757-464c-97dc-81efa8269cd0 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:35.501202 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:35.501889 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.502184 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.502465 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] PATH_INFO: `/v3/projects/9a518cf04eaf44d9a40277d71fb29008` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.503641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a518cf04eaf44d9a40277d71fb29008)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:35.505678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:35.553954 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a518cf04eaf44d9a40277d71fb29008', 'request_id': 'req-14b22797-2e44-4e54-8906-54852384e947'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.561438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a518cf04eaf44d9a40277d71fb29008', 'request_id': 'req-14b22797-2e44-4e54-8906-54852384e947'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.567690 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a518cf04eaf44d9a40277d71fb29008', 'request_id': 'req-14b22797-2e44-4e54-8906-54852384e947'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.577824 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] The token cache is being invalidate because project 9a518cf04eaf44d9a40277d71fb29008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:35.578135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.578762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.580263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1391/2765] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:35 2022] DELETE /identity/v3/projects/9a518cf04eaf44d9a40277d71fb29008 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:35.584941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.594920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cd2b1-2d2a-4473-b844-dc2731219573 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:35.652663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:35.653304 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:35.653634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:35.653943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] PATH_INFO: `/v3/users/adcab6472b4c4ec0a6c27a1ae4179f01` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:35.655108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adcab6472b4c4ec0a6c27a1ae4179f01)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:35.657057 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:35.706298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adcab6472b4c4ec0a6c27a1ae4179f01'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.706999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adcab6472b4c4ec0a6c27a1ae4179f01'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.719723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adcab6472b4c4ec0a6c27a1ae4179f01'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.727296 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adcab6472b4c4ec0a6c27a1ae4179f01'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.734802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adcab6472b4c4ec0a6c27a1ae4179f01'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:35.741127 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1375/2766] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:35 2022] DELETE /identity/v3/users/adcab6472b4c4ec0a6c27a1ae4179f01 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:35.968211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:35.969653 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:35.983336 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22797-2e44-4e54-8906-54852384e947 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.028475 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:36.035741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.037418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.052539 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:36.053833 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.055063 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.055498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.055934 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] PATH_INFO: `/v3/users/95c8c333ddde47448db5fdff3398f792` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.057446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c8c333ddde47448db5fdff3398f792)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.059589 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.108983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd5f4-6f3e-41ad-a81c-223ad6e6beb1 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.109734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c8c333ddde47448db5fdff3398f792'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.113668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.114418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.114700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.114976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] PATH_INFO: `/v3/projects/1c4b06d5bafd40889320955d03049505` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.116144 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4b06d5bafd40889320955d03049505)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.118098 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.122998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c8c333ddde47448db5fdff3398f792'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.130463 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c8c333ddde47448db5fdff3398f792'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.130629 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c8c333ddde47448db5fdff3398f792'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.131099 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fa1def0-664a-497b-9e21-c200638334b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c8c333ddde47448db5fdff3398f792'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.147993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1392/2767] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:35 2022] DELETE /identity/v3/users/95c8c333ddde47448db5fdff3398f792 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:36.153291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.153578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.153880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.168290 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4b06d5bafd40889320955d03049505', 'request_id': 'req-9da7a839-da82-4757-aa11-9daef7510439'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.172907 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:36.175029 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.176193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4b06d5bafd40889320955d03049505', 'request_id': 'req-9da7a839-da82-4757-aa11-9daef7510439'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.182553 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4b06d5bafd40889320955d03049505', 'request_id': 'req-9da7a839-da82-4757-aa11-9daef7510439'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.193776 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] The token cache is being invalidate because project 1c4b06d5bafd40889320955d03049505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:36.194093 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.194753 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-9da7a839-da82-4757-aa11-9daef7510439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.196295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1376/2768] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:36 2022] DELETE /identity/v3/projects/1c4b06d5bafd40889320955d03049505 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:36.203246 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aec6b8-2f1f-42ec-91a8-b188ef5d00a8 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.203537 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aec6b8-2f1f-42ec-91a8-b188ef5d00a8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.203821 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aec6b8-2f1f-42ec-91a8-b188ef5d00a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.223670 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-a3aec6b8-2f1f-42ec-91a8-b188ef5d00a8 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:36.224887 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a3aec6b8-2f1f-42ec-91a8-b188ef5d00a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.236219 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1393/2769] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:36.246431 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.247786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.285402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1377/2770] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:36.293292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.293575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.293863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.306802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57de86-4c40-49f6-bf91-01dd2fc350bd None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.310472 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.311097 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.311376 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.311649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.312993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:36.314239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.321013 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.322205 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1378/2771] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:36.323142 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.326986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.333879 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1394/2772] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:36.336142 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec86e9e-bf24-4065-b88b-2995f8da1e64 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.341511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.342168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.342451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.342725 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.343428 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:36.343859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.345500 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.360896 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1379/2773] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 09:23:36.362043 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.366918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:36.375016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114cd24-a7d9-4236-bd54-6280e1f8d47f service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.382236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.382849 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.383135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.383491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.384453 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.387230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.417668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1395/2774] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:36.421686 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.433058 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e3d7-077a-43d3-8ebf-f018936d6076 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.454980 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:36.501541 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.502583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.502872 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.503153 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.503839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.505779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.529775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1380/2775] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:36.534735 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.545737 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a033-3df1-47d9-b9d7-79f2a54edb58 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.550693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.551324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.551600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.551875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.552591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.554223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.565435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1396/2776] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:36.569159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.580316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ee87-ed9b-4c43-9d80-685f2f71bb0f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.585243 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.585947 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.586300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.586692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.587546 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.589503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.603159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1381/2777] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:36.607561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.611077 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.612734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.621236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1829-339e-41c2-96aa-881dd0aa97ec admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.627580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c361bdd-8e8c-4458-8582-c0dbd29c9c1d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.632859 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.633509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.634000 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.634278 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] PATH_INFO: `/v3/projects/ba2421e2ca164930bfff36c222b85a44` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.636178 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2421e2ca164930bfff36c222b85a44)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.636926 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.637623 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.637946 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.638054 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.638231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] PATH_INFO: `/v3/projects/e96a86a37fc54cf9a3355fd7de25d8f7/users/d87510f0637b4954ab0cd7f9b77a575b/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.653823 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96a86a37fc54cf9a3355fd7de25d8f7, user_id=d87510f0637b4954ab0cd7f9b77a575b, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.656619 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00c2bc-29d0-495f-a865-32fb727cd63d admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.672246 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1397/2778] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:36 2022] PUT /identity/v3/projects/e96a86a37fc54cf9a3355fd7de25d8f7/users/d87510f0637b4954ab0cd7f9b77a575b/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:36.684534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.684968 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.685364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.686991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2421e2ca164930bfff36c222b85a44', 'request_id': 'req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.694920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2421e2ca164930bfff36c222b85a44', 'request_id': 'req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.702216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2421e2ca164930bfff36c222b85a44', 'request_id': 'req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.712853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] The token cache is being invalidate because project ba2421e2ca164930bfff36c222b85a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:36.713132 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.713863 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.715390 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1382/2779] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:36 2022] DELETE /identity/v3/projects/ba2421e2ca164930bfff36c222b85a44 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:36.717223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] MFA Rules not processed for user `d87510f0637b4954ab0cd7f9b77a575b`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:36.718805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.719228 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.720548 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.790041 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2ce8-3ed3-490b-9627-ffca1c11e0a7 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.790829 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1398/2780] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:23:36.795453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.796175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.796521 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.796865 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] PATH_INFO: `/v3/users/f7f83edc430c45549699081a81e8969a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.798210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f83edc430c45549699081a81e8969a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.800236 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.803744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:36.805341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:36.833295 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:36.857105 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f83edc430c45549699081a81e8969a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.858223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f83edc430c45549699081a81e8969a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.872811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f83edc430c45549699081a81e8969a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.873636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824520b2-34bd-4a0f-bc5b-70fe21edf9ac None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:36.877984 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:36.878745 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:36.878892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:36.879321 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:36.883382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f83edc430c45549699081a81e8969a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.889058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:36.890568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:36.894666 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f83edc430c45549699081a81e8969a'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:36.899602 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1399/2781] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:23:36.904702 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1383/2782] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:36 2022] DELETE /identity/v3/users/f7f83edc430c45549699081a81e8969a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:23:37.223028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:37.235087 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda183e5-30cf-40f7-82a6-c3aafacf278e service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:37.278561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:37.305446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:37.306245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.306523 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.306796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] PATH_INFO: `/v3/projects/40ce0fc6215a4cbd917729eb16cc887c` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.308002 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ce0fc6215a4cbd917729eb16cc887c)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:37.309939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:37.361201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ce0fc6215a4cbd917729eb16cc887c', 'request_id': 'req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.369026 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ce0fc6215a4cbd917729eb16cc887c', 'request_id': 'req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.376019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ce0fc6215a4cbd917729eb16cc887c', 'request_id': 'req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.387248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] The token cache is being invalidate because project 40ce0fc6215a4cbd917729eb16cc887c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:37.387541 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.388218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.389788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1400/2783] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:37 2022] DELETE /identity/v3/projects/40ce0fc6215a4cbd917729eb16cc887c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:37.394438 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:37.396904 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:37.410966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ea90f-9bee-44ca-bf7a-bfa6ff5cc477 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:37.468426 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:37.469068 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.469368 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.469660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] PATH_INFO: `/v3/users/f20bfb037437462b9c30b0ea39ea0de9` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.470838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20bfb037437462b9c30b0ea39ea0de9)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:37.472801 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:37.520733 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20bfb037437462b9c30b0ea39ea0de9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.521386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20bfb037437462b9c30b0ea39ea0de9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.535485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20bfb037437462b9c30b0ea39ea0de9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.543036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20bfb037437462b9c30b0ea39ea0de9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.549805 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-82da2540-1d59-4be8-bcaa-f6ec1d73cff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20bfb037437462b9c30b0ea39ea0de9'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.555710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1384/2784] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:37 2022] DELETE /identity/v3/users/f20bfb037437462b9c30b0ea39ea0de9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:37.830565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:37.832082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:37.845417 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b81bc-7f09-4e0c-a1c5-d6b96d905fff admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:37.906887 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:37.907535 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:37.907816 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:37.908091 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] PATH_INFO: `/v3/projects/c79fc7dca8514cc5bc56bd60797978f5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:37.909267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79fc7dca8514cc5bc56bd60797978f5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:37.911364 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:37.960429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79fc7dca8514cc5bc56bd60797978f5', 'request_id': 'req-4431276a-28d7-4e14-9a6f-994d12a9030e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.973652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79fc7dca8514cc5bc56bd60797978f5', 'request_id': 'req-4431276a-28d7-4e14-9a6f-994d12a9030e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.981371 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79fc7dca8514cc5bc56bd60797978f5', 'request_id': 'req-4431276a-28d7-4e14-9a6f-994d12a9030e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.989209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:37.994149 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] The token cache is being invalidate because project c79fc7dca8514cc5bc56bd60797978f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:37.995578 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.996189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-4431276a-28d7-4e14-9a6f-994d12a9030e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:37.997998 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1401/2785] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:37 2022] DELETE /identity/v3/projects/c79fc7dca8514cc5bc56bd60797978f5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:38.004038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.004332 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.004612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.024742 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:38.026042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:38.045217 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.097067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1385/2786] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:38.098716 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.099096 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.100113 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.119577 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:38.121341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:38.129422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1402/2787] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:38.133408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.143764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff692cc-970f-494e-9048-120b5a9990ac None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.148418 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.149036 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.149316 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.149590 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.150325 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:38.150629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.152253 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.163697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1386/2788] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:38.168146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.177340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644f400-6ddc-4be7-91cb-3e4e88eb086f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.181997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.183118 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.183395 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.183779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.184435 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.186351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.214467 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1403/2789] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:38.218743 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.228784 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2efa0d-d30a-4a1b-b649-80b85db3c8e5 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.289067 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.289853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.290150 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.290452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.291140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.293601 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.316623 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1387/2790] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:38.320561 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.331570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2c09-014f-4f4f-b3ab-1f74b0e54845 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.336995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.337928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.338274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.338618 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.339414 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.341066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.351583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1404/2791] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:38.357804 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.368387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74554-14ae-4869-a151-ac650ae5fd16 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.375216 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.375901 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.376483 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.376723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.377511 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.379168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314faa4-d8d8-4c0b-8bc6-c7f2cddc8321 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.390716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1388/2792] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:38.395268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.406788 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59545-59d9-4a26-81f4-2d0d7d70d8ae admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.411656 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.412330 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.413178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.413178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] PATH_INFO: `/v3/projects/135660866d51491c8f7bd18f6ade85f7/users/9fade52251834e1190435c6c6588567e/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.428784 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] RBAC: Authorizing `identity:create_grant(project_id=135660866d51491c8f7bd18f6ade85f7, user_id=9fade52251834e1190435c6c6588567e, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.431512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.446285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1405/2793] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:38 2022] PUT /identity/v3/projects/135660866d51491c8f7bd18f6ade85f7/users/9fade52251834e1190435c6c6588567e/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:38.457417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.457892 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.458015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.492184 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] MFA Rules not processed for user `9fade52251834e1190435c6c6588567e`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:38.493423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:38.557451 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1389/2794] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:38.562627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.622979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a13cf-c347-4c85-b824-d723ebfb895e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.627543 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.628155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.628438 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.628715 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.629368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.631299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.657940 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1406/2795] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:38.662014 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.673341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fb4c1-3268-49fc-8eae-7972cb40d954 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.685565 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:38.732577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.733190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.733468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.733741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.734420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.736328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.759486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1390/2796] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:38.763583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.773774 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041976-ae73-4610-8c35-197378fde026 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.779111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.779722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.779995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.780267 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.780957 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.782579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.793747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1407/2797] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:38.798099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.808111 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135795-e467-439c-a56e-69d4cc9d4781 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.812697 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.813362 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.813647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.813955 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] PATH_INFO: `/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609/users/a5c18b3998824ced8be6cbb1560ef383/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.828417 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd66d0bb6c24c4db83fc504cdeb2609, user_id=a5c18b3998824ced8be6cbb1560ef383, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.831160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.844739 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1391/2798] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:38 2022] PUT /identity/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609/users/a5c18b3998824ced8be6cbb1560ef383/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:38.848551 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.858910 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33baa37-1768-49b2-8ae4-4ebda2da667a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.914454 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.914990 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.915268 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.915540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.916252 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.918510 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2e6f3-168d-4964-b39d-4792607fe5dc admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.928508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1408/2799] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:38.932707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:38.943171 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30129e-7c05-42bb-a7de-e12c4dfc7bdf admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:38.947778 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:38.948440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.948719 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.948994 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] PATH_INFO: `/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609/users/a5c18b3998824ced8be6cbb1560ef383/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.950943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd66d0bb6c24c4db83fc504cdeb2609, user_id=a5c18b3998824ced8be6cbb1560ef383, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:38.953575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:38.964397 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1392/2800] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:38 2022] PUT /identity/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609/users/a5c18b3998824ced8be6cbb1560ef383/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:38.976058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:38.976340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:38.976638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:38.985388 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:39.005611 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] MFA Rules not processed for user `a5c18b3998824ced8be6cbb1560ef383`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:39.006846 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:39.073802 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1409/2801] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:39.085203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:39.086986 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:39.144989 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5e69d-e0a1-4d3d-aa79-695ec01f373c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:39.148820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:39.149430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:39.149706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:39.150006 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:39.160941 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:39.162591 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7671a-9801-48fa-8b95-010f5ff4cd6f service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:39.176155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1393/2802] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:39.548933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:39.610282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fe199-07a1-4d33-91ed-4e336d580c67 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:39.614939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:39.615576 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:39.615895 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:39.616167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] PATH_INFO: `/v3/users/9fade52251834e1190435c6c6588567e` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:39.617381 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fade52251834e1190435c6c6588567e)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:39.619354 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:39.663528 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fade52251834e1190435c6c6588567e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:39.675323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fade52251834e1190435c6c6588567e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:39.682069 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fade52251834e1190435c6c6588567e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:39.683263 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fade52251834e1190435c6c6588567e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:39.683867 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fade52251834e1190435c6c6588567e'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:39.695851 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1410/2803] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:39 2022] DELETE /identity/v3/users/9fade52251834e1190435c6c6588567e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:39.700876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df9d06-39f0-4912-a223-16d263012598 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:39.701401 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df9d06-39f0-4912-a223-16d263012598 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:39.701516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df9d06-39f0-4912-a223-16d263012598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:39.720310 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-97df9d06-39f0-4912-a223-16d263012598 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:39.721531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-97df9d06-39f0-4912-a223-16d263012598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:39.779956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1394/2804] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:39.789728 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:39.791162 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:39.803167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:39.851157 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cab6-d212-4bba-8c2c-962bd731f497 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:39.855282 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:39.855892 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:39.856169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:39.856441 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:39.865323 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:39.866779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:39.876592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1411/2805] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:40.145040 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df9d06-39f0-4912-a223-16d263012598 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:40.146486 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-97df9d06-39f0-4912-a223-16d263012598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:40.159746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df9d06-39f0-4912-a223-16d263012598 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:40.164777 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:40.165424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.165705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.166010 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] PATH_INFO: `/v3/projects/135660866d51491c8f7bd18f6ade85f7` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.167765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135660866d51491c8f7bd18f6ade85f7)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:40.169668 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:40.178764 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:40.218008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135660866d51491c8f7bd18f6ade85f7', 'request_id': 'req-be3f9805-2ccb-477d-8317-c9e9341aeb69'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.225616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135660866d51491c8f7bd18f6ade85f7', 'request_id': 'req-be3f9805-2ccb-477d-8317-c9e9341aeb69'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.232484 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135660866d51491c8f7bd18f6ade85f7', 'request_id': 'req-be3f9805-2ccb-477d-8317-c9e9341aeb69'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.242972 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] The token cache is being invalidate because project 135660866d51491c8f7bd18f6ade85f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:40.243254 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.243895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.245430 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1395/2806] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:40 2022] DELETE /identity/v3/projects/135660866d51491c8f7bd18f6ade85f7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:40.249337 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:40.259070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c080ae-1869-47ac-a515-4a0c807a78dd service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:40.315290 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:40.315918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.316198 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.316473 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] PATH_INFO: `/v3/users/a5c18b3998824ced8be6cbb1560ef383` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.317624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c18b3998824ced8be6cbb1560ef383)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:40.319567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:40.368497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c18b3998824ced8be6cbb1560ef383'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.380741 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c18b3998824ced8be6cbb1560ef383'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.388188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c18b3998824ced8be6cbb1560ef383'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.389006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c18b3998824ced8be6cbb1560ef383'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.389624 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-019cd678-cab3-4028-8c78-c094df50fe76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c18b3998824ced8be6cbb1560ef383'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.403595 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1412/2807] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:23:40 2022] DELETE /identity/v3/users/a5c18b3998824ced8be6cbb1560ef383 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:40.684293 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:40.685812 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:40.701135 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9805-2ccb-477d-8317-c9e9341aeb69 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:40.757811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:40.758413 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.758693 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.758968 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] PATH_INFO: `/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.760120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd66d0bb6c24c4db83fc504cdeb2609)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:40.762584 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:40.807197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd66d0bb6c24c4db83fc504cdeb2609', 'request_id': 'req-17997dcf-8f0e-4548-afde-dd834b9a8a98'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.814162 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd66d0bb6c24c4db83fc504cdeb2609', 'request_id': 'req-17997dcf-8f0e-4548-afde-dd834b9a8a98'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.820207 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd66d0bb6c24c4db83fc504cdeb2609', 'request_id': 'req-17997dcf-8f0e-4548-afde-dd834b9a8a98'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.831072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] The token cache is being invalidate because project abd66d0bb6c24c4db83fc504cdeb2609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:23:40.831355 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.832009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-17997dcf-8f0e-4548-afde-dd834b9a8a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:23:40.833509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1396/2808] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:23:40 2022] DELETE /identity/v3/projects/abd66d0bb6c24c4db83fc504cdeb2609 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:40.839562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.840065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.840065 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.859188 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:40.860385 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:40.885739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:40.920527 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1413/2809] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:40.928524 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.928809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.929137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.949639 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:40.950853 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:40.958688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1397/2810] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:40.963154 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:40.972797 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b6ef0-472b-4f38-be2f-ea86852a2f8f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:40.977975 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:40.978591 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:40.978868 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:40.979143 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:40.979837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:23:40.980141 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:40.981744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:40.993090 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1414/2811] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:23:40.997785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.007078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ad82f-cbf3-4dfa-8c4e-2ec57892e837 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.012348 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.012937 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.013199 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.013467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.014239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.016309 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.036560 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:41.044493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1398/2812] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:23:40 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:23:41.048404 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.058976 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95624a14-1719-4a4b-8bbe-c248b1797f7b admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.113793 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.114444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.114725 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.115001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.115673 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.117594 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.139410 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1415/2813] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:23:41 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:23:41.143492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.154122 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ae1-36e5-419b-a700-e58ab3d2cc14 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.158773 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.159378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.159655 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.159927 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.160612 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.162230 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.172712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1399/2814] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:41.176657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.187639 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801c7f-3ce3-435e-b883-0fb10e73b204 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.192411 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.193023 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.193302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.193599 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.194324 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.195904 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589014f-326c-4b01-af29-cbdf19443ef8 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.207000 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1416/2815] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:23:41.211262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.221275 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a67fd-d60f-4855-96d9-418eb53115c3 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.225982 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.226642 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.226919 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.227189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] PATH_INFO: `/v3/projects/5f11af5630c44d54b7d9e9c9b6634ae1/users/34c3472d9b8147b5a1ba6b825ba02b56/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.242361 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f11af5630c44d54b7d9e9c9b6634ae1, user_id=34c3472d9b8147b5a1ba6b825ba02b56, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.245002 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.258229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1400/2816] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:23:41 2022] PUT /identity/v3/projects/5f11af5630c44d54b7d9e9c9b6634ae1/users/34c3472d9b8147b5a1ba6b825ba02b56/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:23:41.269757 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.270028 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.270326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.298875 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] MFA Rules not processed for user `34c3472d9b8147b5a1ba6b825ba02b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:23:41.300100 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:41.361856 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1417/2817] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:23:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:23:41.374051 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:41.375756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:41.434224 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13d971-039c-4940-a41d-586069c9f572 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:41.438229 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:41.438850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:41.439130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:41.439404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:41.447529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:41.448957 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:41.457581 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1401/2818] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:49.974644 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:49.976191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:50.007158 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:23:50.035615 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0c949-c920-4557-81c1-500d9d9c2357 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:50.039287 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:50.039912 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:50.040193 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:50.040466 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:50.042231 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:50.052822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:50.054314 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:50.062514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1418/2819] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:50.184055 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:50.186376 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:50.199201 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba9cb-ab63-4408-bb28-10823fe8e48f service glance] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:50.202828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:50.203503 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:50.203716 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:50.203990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:50.205889 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:50.217019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:50.218563 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:50.228038 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1402/2820] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:23:50.310648 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:23:50.319422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6861f-5490-4bf7-a560-45dafe22cbfd service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:23:50.323375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:23:50.324542 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:23:50.324819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:23:50.325089 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:23:50.326924 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:23:50.399635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:23:50.401068 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93489593-e90c-4cf8-96e7-c0eadf500817 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:23:50.412649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1419/2821] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:23:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:00.224249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=48,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:00.407768 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=48 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:00.493358 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:00.494803 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:00.560983 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97bf02-d365-43c8-907b-cc9c5be47114 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:00.565164 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:00.565810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:00.566177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:00.566455 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] PATH_INFO: `/v3/users/d87510f0637b4954ab0cd7f9b77a575b` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:00.567727 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87510f0637b4954ab0cd7f9b77a575b)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:00.570048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:00.616810 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87510f0637b4954ab0cd7f9b77a575b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:00.617463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87510f0637b4954ab0cd7f9b77a575b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:00.629651 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87510f0637b4954ab0cd7f9b77a575b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:00.636794 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87510f0637b4954ab0cd7f9b77a575b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:00.644152 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87510f0637b4954ab0cd7f9b77a575b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:00.650168 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1403/2822] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:24:00 2022] DELETE /identity/v3/users/d87510f0637b4954ab0cd7f9b77a575b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:00.654948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:00.655238 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:00.655518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:00.675852 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:00.677046 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:00.737640 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1420/2823] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:00.747219 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:00.748631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:00.807876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e69f2-8512-4d6f-8ff5-10cf901614a2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:00.811575 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:00.812189 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:00.812493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:00.812765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:00.821724 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:00.823191 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:00.833047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1404/2824] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:01.130955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:01.132419 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:01.145049 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9c11-6e0e-4154-99f0-c50f6bc10964 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:01.149786 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:01.150462 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:01.150746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:01.151021 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] PATH_INFO: `/v3/projects/e96a86a37fc54cf9a3355fd7de25d8f7` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:01.152200 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96a86a37fc54cf9a3355fd7de25d8f7)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:01.154150 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:01.202316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96a86a37fc54cf9a3355fd7de25d8f7', 'request_id': 'req-8b030a8d-d5f0-4c06-89db-b3d868f46226'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:01.210657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96a86a37fc54cf9a3355fd7de25d8f7', 'request_id': 'req-8b030a8d-d5f0-4c06-89db-b3d868f46226'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:01.217136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96a86a37fc54cf9a3355fd7de25d8f7', 'request_id': 'req-8b030a8d-d5f0-4c06-89db-b3d868f46226'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:01.227449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] The token cache is being invalidate because project e96a86a37fc54cf9a3355fd7de25d8f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:24:01.227739 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:01.228375 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:01.229909 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1421/2825] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:24:01 2022] DELETE /identity/v3/projects/e96a86a37fc54cf9a3355fd7de25d8f7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:10.829016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:11.224512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:31.563478 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:31.564869 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:31.636594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123ae-dc9c-45ed-a358-da80404d2c6b service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:31.698574 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:31.699305 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:31.699490 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:31.699875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:31.701816 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:31.714019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:31.715618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c774c1-bc87-4f4f-9e09-2da0333cedc5 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:31.726577 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1405/2826] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:32.751011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:32.752423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:32.815129 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b030a8d-d5f0-4c06-89db-b3d868f46226 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:32.819696 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:32.820344 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:32.820627 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:32.821512 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] PATH_INFO: `/v3/users/34c3472d9b8147b5a1ba6b825ba02b56` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:32.822671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c3472d9b8147b5a1ba6b825ba02b56)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:32.824605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:32.871014 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c3472d9b8147b5a1ba6b825ba02b56'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:32.882636 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c3472d9b8147b5a1ba6b825ba02b56'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:32.889997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c3472d9b8147b5a1ba6b825ba02b56'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:32.890668 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c3472d9b8147b5a1ba6b825ba02b56'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:32.891271 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c3472d9b8147b5a1ba6b825ba02b56'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:32.903597 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1422/2827] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:24:32 2022] DELETE /identity/v3/users/34c3472d9b8147b5a1ba6b825ba02b56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:32.908256 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:32.908741 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:32.908871 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:32.932123 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:32.933345 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:32.993024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1406/2828] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:33.002558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.003948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:33.060372 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c72e-d97c-47b2-a3cb-00a3ed59caa0 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.064018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.064638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.064916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.065240 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.074123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.075586 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dba128-c083-4074-a4ae-f7828485d5d5 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.084781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1423/2829] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:33.501436 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.503005 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:33.515075 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35598d9d-70b4-403e-b710-c737fc721308 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.519722 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.520346 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.520630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.520903 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] PATH_INFO: `/v3/projects/5f11af5630c44d54b7d9e9c9b6634ae1` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.522634 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f11af5630c44d54b7d9e9c9b6634ae1)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.524580 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.558673 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:33.574423 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f11af5630c44d54b7d9e9c9b6634ae1', 'request_id': 'req-b218ceb6-6381-40d5-b79b-e067562bac6c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:33.582223 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f11af5630c44d54b7d9e9c9b6634ae1', 'request_id': 'req-b218ceb6-6381-40d5-b79b-e067562bac6c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:33.589671 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f11af5630c44d54b7d9e9c9b6634ae1', 'request_id': 'req-b218ceb6-6381-40d5-b79b-e067562bac6c'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:33.600204 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] The token cache is being invalidate because project 5f11af5630c44d54b7d9e9c9b6634ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:24:33.600491 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:33.601170 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-b218ceb6-6381-40d5-b79b-e067562bac6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:33.603607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1407/2830] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:24:33 2022] DELETE /identity/v3/projects/5f11af5630c44d54b7d9e9c9b6634ae1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:33.608127 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.608418 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.608698 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.627660 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:33.628877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:33.645413 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:33.691780 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1424/2831] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:33.700285 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.700707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.701935 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.722396 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:33.723611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:33.735292 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1408/2832] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:33.741123 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.751511 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808d595-5a3a-4b12-a45a-0ae840a9dc4f None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.756313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.756935 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.757214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.757487 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.758341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80375) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:33.758645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.760241 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.772057 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1425/2833] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:33.777107 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.786381 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d1766-518c-477e-8938-dd0d1ae4c1db None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.791734 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.792353 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.792632 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.792926 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.793582 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.795529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.823012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1409/2834] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:24:33 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:24:33.827273 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.838053 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df159-cba7-47d2-bc5a-28a7a70678c4 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.896206 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.896821 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.897116 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.897373 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] PATH_INFO: `/v3/users` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.898691 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.901351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.926914 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1426/2835] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:24:33 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:24:33.930616 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.941072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f19a7a-65c1-4405-b5af-3a14201fd8ad admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.946366 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.947158 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.947611 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.947890 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.948587 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.950225 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.960125 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1410/2836] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:24:33.964787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:33.974645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408644fb-c566-42aa-a79f-10bae78c056c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:33.979515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:33.980514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:33.980925 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:33.981292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:33.982613 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:33.984927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4aa67-4936-4bc1-8103-4455637d66f5 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:33.996244 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1427/2837] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:24:34.000576 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:34.011501 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793e0ea-d25d-4001-91e9-18217f2de907 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:34.016261 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:34.016918 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:34.017208 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:34.017489 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] PATH_INFO: `/v3/projects/7ea63f58bc674db7b5ce6274a3785897/users/8941b7a73e7948e0a243c3356e151809/roles/6273caafd8e64714b134898490672923` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:34.032453 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea63f58bc674db7b5ce6274a3785897, user_id=8941b7a73e7948e0a243c3356e151809, role_id=6273caafd8e64714b134898490672923)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:34.035140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:34.048084 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1411/2838] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:24:33 2022] PUT /identity/v3/projects/7ea63f58bc674db7b5ce6274a3785897/users/8941b7a73e7948e0a243c3356e151809/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:34.059457 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e3c7-f21b-4be7-a55c-6536f4443aed None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:34.059748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e3c7-f21b-4be7-a55c-6536f4443aed None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:34.060022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e3c7-f21b-4be7-a55c-6536f4443aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:34.089480 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-ffc2e3c7-f21b-4be7-a55c-6536f4443aed None None] MFA Rules not processed for user `8941b7a73e7948e0a243c3356e151809`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:34.090641 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-ffc2e3c7-f21b-4be7-a55c-6536f4443aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:34.153787 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1428/2839] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:24:34.164498 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:34.166235 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:34.223911 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ec81-f1de-434a-86e0-a411d5b6f95f admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:34.227593 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:34.228210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:34.228493 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:34.228769 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:34.237130 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:34.238596 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e95765-d02a-4e2c-84fe-88a766ff3f07 service cinder] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:34.248239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1412/2840] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:35.608521 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:35.609047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:35.609047 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:35.628540 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] MFA Rules not processed for user `8941b7a73e7948e0a243c3356e151809`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:35.629781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:35.636819 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1429/2841] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:44.244220 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:45.623111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:50.903016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d02a-91fd-45d1-a5e1-84ff972bd7f8 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:50.903522 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d02a-91fd-45d1-a5e1-84ff972bd7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:50.903628 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d02a-91fd-45d1-a5e1-84ff972bd7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:50.924249 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-c487d02a-91fd-45d1-a5e1-84ff972bd7f8 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:50.925527 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c487d02a-91fd-45d1-a5e1-84ff972bd7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:50.992981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1413/2842] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:51.041495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.043203 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:51.139762 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7e093-3023-4f2e-ae1e-4270badec249 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.144813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.145446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.145722 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.146022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.157039 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:51.158693 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:51.167993 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1430/2843] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:24:51.742250 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a871b5-e87c-4272-b5fa-b7579af5b5ce None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.742542 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a871b5-e87c-4272-b5fa-b7579af5b5ce None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.742850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a871b5-e87c-4272-b5fa-b7579af5b5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.764439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-52a871b5-e87c-4272-b5fa-b7579af5b5ce None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:51.765614 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-52a871b5-e87c-4272-b5fa-b7579af5b5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:51.773588 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1414/2844] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:51.786310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:51.798156 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1583b22-71a0-4e9a-9972-9fe05f47fd69 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:51.803342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:51.804146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:51.804391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:51.804497 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:51.813699 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:51.815335 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:51.824596 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1431/2845] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:24:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:24:52.051024 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878078-d657-4518-bccb-cdada9f77c50 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.051515 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878078-d657-4518-bccb-cdada9f77c50 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.051624 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878078-d657-4518-bccb-cdada9f77c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.076408 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-99878078-d657-4518-bccb-cdada9f77c50 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:52.077607 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-99878078-d657-4518-bccb-cdada9f77c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:52.084771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1415/2846] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:24:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:52.098349 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.109351 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2aa9-6e66-4765-8160-89959d567f94 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.114652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.115280 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.115558 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.115828 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.125429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.127081 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.136370 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1432/2847] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:24:52.503350 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.503645 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.503940 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.523699 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:52.525019 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:52.532913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1416/2848] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:24:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:52.545592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:52.558171 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5f14-439e-47c8-935a-f2383ae2b786 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:52.564979 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:52.565657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:52.565928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:52.566340 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:52.577366 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:52.579055 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809520-fc90-4952-9ee2-19b9288eac47 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:52.588074 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1433/2849] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:24:54.872635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:54.942011 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba62755-aee4-4edf-b429-24711a9f8237 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:54.946759 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:54.947419 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:54.947701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:54.947976 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] PATH_INFO: `/v3/users/8941b7a73e7948e0a243c3356e151809` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:54.949688 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8941b7a73e7948e0a243c3356e151809)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:54.951732 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.001085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8941b7a73e7948e0a243c3356e151809'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.001744 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8941b7a73e7948e0a243c3356e151809'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.014990 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8941b7a73e7948e0a243c3356e151809'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.023313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8941b7a73e7948e0a243c3356e151809'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.031375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8941b7a73e7948e0a243c3356e151809'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.037531 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1417/2850] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:24:54 2022] DELETE /identity/v3/users/8941b7a73e7948e0a243c3356e151809 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:55.042604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.043103 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.043205 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.063339 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:55.064582 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.126313 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1434/2851] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:55.136110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.137525 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.144193 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:55.202414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a09105-2faf-46c3-8f72-90afca99cc8c admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.206117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.206747 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.207025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.207298 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.217238 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.218695 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d025e5-ea72-4667-9929-5f3d903f7171 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.228085 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1418/2852] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:24:55.413547 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.415685 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.429638 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c958d85-d034-4134-9b1c-9fbe1701a00a None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.435444 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.436105 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.436384 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.436657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] PATH_INFO: `/v3/projects/7ea63f58bc674db7b5ce6274a3785897` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.438637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea63f58bc674db7b5ce6274a3785897)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.440610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.481041 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:55.492145 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea63f58bc674db7b5ce6274a3785897', 'request_id': 'req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.500291 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea63f58bc674db7b5ce6274a3785897', 'request_id': 'req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.506776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea63f58bc674db7b5ce6274a3785897', 'request_id': 'req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.517744 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] The token cache is being invalidate because project 7ea63f58bc674db7b5ce6274a3785897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:24:55.518059 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.518749 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-49c10d92-c6a7-44fb-bc6d-d48b12fddf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:24:55.520285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1435/2853] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:24:55 2022] DELETE /identity/v3/projects/7ea63f58bc674db7b5ce6274a3785897 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:24:55.525707 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.526020 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.526336 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.548806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:55.550047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.611885 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1419/2854] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:55.621422 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.621692 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.622003 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.643048 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:55.644285 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:55.653669 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1436/2855] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:55.658322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.668711 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2fbf5-0a3f-498a-b6ac-bd9774c3c593 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.673660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.674365 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.674679 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.674956 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.675705 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80374) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 09:24:55.676015 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.677675 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.691009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1420/2856] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Jul 28 09:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 09:24:55.695625 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.706403 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c63af-afb4-4b77-b70f-cd54a627b921 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.711655 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.712289 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.712565 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.712837 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.713508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.716209 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.745233 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1437/2857] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:24:55.749647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.761239 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82216-dc92-43b8-a2fb-ac6c6b00cb29 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.821053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.821700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.822012 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.822277 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.822963 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.825475 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.852663 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1421/2858] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 09:24:55.854007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.865942 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe7d5-1c35-4656-9016-bb2ada4ba3eb admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.872927 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.874058 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.874562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.874995 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.875748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.877423 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.889115 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1438/2859] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:24:55.893119 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.903630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462b0d-9ba4-4dfd-9afc-aa7de63e52f2 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.908461 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.909025 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.909307 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.909583 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.910364 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.911984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632594-d782-49f0-9902-9f4f8e0c84c1 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.923683 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1422/2860] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:24:55.928178 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:55.938876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df4c7e-9abd-4848-b3c3-cff0774ee56c admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:55.943652 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:55.944325 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.944603 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.944876 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] PATH_INFO: `/v3/projects/33b2e26d27c84e738eede31dc3c9ee3a/users/cafc77c67a844f248b0879d39d5511dd/roles/6273caafd8e64714b134898490672923` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:55.966020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b2e26d27c84e738eede31dc3c9ee3a, user_id=cafc77c67a844f248b0879d39d5511dd, role_id=6273caafd8e64714b134898490672923)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:55.968806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:55.984667 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1439/2861] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:24:55 2022] PUT /identity/v3/projects/33b2e26d27c84e738eede31dc3c9ee3a/users/cafc77c67a844f248b0879d39d5511dd/roles/6273caafd8e64714b134898490672923 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:24:55.990433 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ca1d-7979-4589-86b1-e6f069c5f904 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:55.990718 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ca1d-7979-4589-86b1-e6f069c5f904 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:55.990996 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ca1d-7979-4589-86b1-e6f069c5f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.012072 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-ba31ca1d-7979-4589-86b1-e6f069c5f904 None None] MFA Rules not processed for user `7b84f6da05694fb8ade9ceaae63ccb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:24:56.013311 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ba31ca1d-7979-4589-86b1-e6f069c5f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:56.025630 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:24:56.080922 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1423/2862] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:24:56.091769 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:24:56.093538 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:24:56.157748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54544f8-59a5-4f81-9b7f-68ddf62b2f76 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:24:56.161498 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:24:56.162101 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:24:56.162382 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:24:56.162657 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:24:56.172452 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:24:56.173929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:24:56.183631 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1440/2863] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:25:00.666344 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:00.667594 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:00.667917 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:00.718001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] MFA Rules not processed for user `cafc77c67a844f248b0879d39d5511dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:00.721045 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:00.826713 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1424/2864] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:25:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 09:25:00.831022 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:00.842429 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac218f-a073-4d78-b306-7929eb1923f8 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:00.847747 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:00.848678 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:00.848960 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:00.849415 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:00.850136 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:00.852079 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:00.903391 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1441/2865] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Jul 28 09:25:00 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:25:00.909175 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:00.933485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a82f1-4075-4038-b570-4072890dcc59 None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.008378 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.009117 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.009476 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.009823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] PATH_INFO: `/v3/users` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.010677 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.012698 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.041042 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1425/2866] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Jul 28 09:25:00 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 09:25:01.042901 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.055515 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a83541-3133-480e-9cec-e1f1a87e9ae2 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.061201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.061587 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.061880 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.062605 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.062938 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.064567 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.077034 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1442/2867] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:25:01.082341 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.100765 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb94bd-1968-48ce-9729-9fff02b838fe admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.107328 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.108022 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.108303 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.108579 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] PATH_INFO: `/v3/projects/434939b146894b19bbed606c6776b66c/users/0095212a29754d75bea38b7f8819983f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.125440 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434939b146894b19bbed606c6776b66c, user_id=0095212a29754d75bea38b7f8819983f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.128190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.141756 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1426/2868] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:25:01 2022] PUT /identity/v3/projects/434939b146894b19bbed606c6776b66c/users/0095212a29754d75bea38b7f8819983f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:25:01.148862 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.162332 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7bbfd-0c5b-45fd-86e3-1ab8a868c4c6 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.192393 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.246908 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.247570 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.247845 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.248117 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.248928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.250643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7e3c0-dcde-454e-a5ee-e49f1d43bf74 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.263070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1443/2869] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Jul 28 09:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 09:25:01.268700 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:01.280895 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e9f4-a64d-443d-b15f-94037dacc606 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:01.285761 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:01.286492 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:01.286775 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:01.287053 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] PATH_INFO: `/v3/projects/434939b146894b19bbed606c6776b66c/users/0095212a29754d75bea38b7f8819983f/roles/6f1ebd186c9845f39f6451326a6fcbac` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:01.289070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434939b146894b19bbed606c6776b66c, user_id=0095212a29754d75bea38b7f8819983f, role_id=6f1ebd186c9845f39f6451326a6fcbac)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:01.291793 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:01.296003 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:01.308464 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1427/2870] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Jul 28 09:25:01 2022] PUT /identity/v3/projects/434939b146894b19bbed606c6776b66c/users/0095212a29754d75bea38b7f8819983f/roles/6f1ebd186c9845f39f6451326a6fcbac => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:25:06.080468 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719bf4-dcf3-43ce-9780-c6d2c481f655 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.081011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719bf4-dcf3-43ce-9780-c6d2c481f655 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.081011 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719bf4-dcf3-43ce-9780-c6d2c481f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.126302 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-31719bf4-dcf3-43ce-9780-c6d2c481f655 None None] MFA Rules not processed for user `0095212a29754d75bea38b7f8819983f`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:06.127568 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-31719bf4-dcf3-43ce-9780-c6d2c481f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:06.207309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1444/2871] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:25:06.220943 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:06.222758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:06.323297 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51d07-e125-4ba6-b7aa-62dca2998b14 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:06.327126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e8151c2541400cb76b8b19384f49d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:06.327858 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.328263 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.328603 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.390752 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:06.392222 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:06.403016 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1428/2872] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:25:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:25:06.537748 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3fb08-96b9-45a7-bdc1-9ea979ea5de8 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.538299 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3fb08-96b9-45a7-bdc1-9ea979ea5de8 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.538583 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3fb08-96b9-45a7-bdc1-9ea979ea5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.560734 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-04f3fb08-96b9-45a7-bdc1-9ea979ea5de8 None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:06.562006 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-04f3fb08-96b9-45a7-bdc1-9ea979ea5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:06.651877 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1445/2873] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:25:06.664078 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:06.666467 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:06.749255 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b32a-f0cb-41b9-a219-238eecce3c09 service glance] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:06.755443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:06.756351 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:06.756629 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:06.757154 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:06.766915 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:06.768495 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522cba6-4a91-4aab-812b-231f5d6385d5 service swift] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:06.778981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1429/2874] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:25:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 09:25:16.644820 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=75 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:16.671390 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:16.671675 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:16.671951 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:16.691155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] MFA Rules not processed for user `cafc77c67a844f248b0879d39d5511dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:16.692449 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:16.699705 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1446/2875] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:25:16.774155 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=91 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:18.817089 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:18.817691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:18.818647 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:18.844771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:25:18.846059 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:18.853641 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1430/2876] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Jul 28 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:25:18.870545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:18.882514 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d7c1-fac7-4f1b-bcae-c6a317fbdb11 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:18.888189 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:18.888882 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:18.889245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:18.889588 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:18.900020 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:18.901806 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:18.911363 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1447/2877] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 09:25:28.837386 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:28.907310 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=9 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:25:47.292720 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:47.353984 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab64eca-d243-4c45-9745-837f31a9c4fd None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:47.357691 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:47.358374 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:47.358652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:47.358920 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:47.360820 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:47.436241 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:47.437768 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:47.450097 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1431/2878] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:25:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:25:51.305813 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:51.309534 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:51.324331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ca5e0-5501-42fa-bd8f-28212fbf6fc7 service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:51.328260 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:51.328966 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:51.329635 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:51.329955 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:51.344082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:51.346032 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:51.358447 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1448/2879] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:25:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:25:52.936540 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:52.954886 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5279c7-2eee-4d5c-adfb-e0a0f5eb5453 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:52.959121 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:52.959997 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:52.960283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:52.960559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:52.962876 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:53.043785 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:53.045233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:53.055845 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1432/2880] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:25:53.436269 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:25:53.438399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:53.549316 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297211f2-80ea-4ed2-a258-12f617487594 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:25:53.558671 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:25:53.559451 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:25:53.559760 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:25:53.560036 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:25:53.563947 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:25:53.579796 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:25:53.581871 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:25:53.595592 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1449/2881] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:25:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:26:01.497021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:01.498479 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:01.550848 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=100 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:01.565070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:01.567297 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:01.575470 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b0c6-b4c1-41bf-9911-d020b1fbc75a service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:01.585627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:01.586313 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:01.586585 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:01.586867 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:01.589414 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:01.593092 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923fd23-9cc9-4761-905b-608b29a8ee84 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:01.598288 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:01.598637 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:01.598996 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:01.599342 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] PATH_INFO: `/v3/users/bcc41a77cdfb426caa5c86ecab2298e8` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:01.600600 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc41a77cdfb426caa5c86ecab2298e8)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:01.601746 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:01.602604 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:01.604070 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:01.615252 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1433/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:26:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:26:01.662980 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc41a77cdfb426caa5c86ecab2298e8'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:01.677226 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc41a77cdfb426caa5c86ecab2298e8'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:01.685962 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc41a77cdfb426caa5c86ecab2298e8'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:01.686755 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc41a77cdfb426caa5c86ecab2298e8'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:01.687493 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc41a77cdfb426caa5c86ecab2298e8'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:01.708019 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1450/2883] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:26:01 2022] DELETE /identity/v3/users/bcc41a77cdfb426caa5c86ecab2298e8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:26:01.887509 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:01.911300 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8aa61-994c-4d2e-86b7-5f4f5ef01f75 service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:01.971482 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:01.972145 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:01.972424 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:01.972710 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] PATH_INFO: `/v3/projects/53fd239676ec42e0a9f145e924fc249a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:01.973857 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53fd239676ec42e0a9f145e924fc249a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:01.975790 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:02.021231 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53fd239676ec42e0a9f145e924fc249a', 'request_id': 'req-ca3738d6-aac9-423f-b00e-061b2e3e617d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.029347 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53fd239676ec42e0a9f145e924fc249a', 'request_id': 'req-ca3738d6-aac9-423f-b00e-061b2e3e617d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.036029 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53fd239676ec42e0a9f145e924fc249a', 'request_id': 'req-ca3738d6-aac9-423f-b00e-061b2e3e617d'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.046547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] The token cache is being invalidate because project 53fd239676ec42e0a9f145e924fc249a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:26:02.046829 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.047452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.048939 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1434/2884] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:26:01 2022] DELETE /identity/v3/projects/53fd239676ec42e0a9f145e924fc249a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:26:02.053399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:02.054518 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:02.068207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b58ec6-bac4-4560-9a55-f718f1bee4ca admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:02.109562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:02.125920 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:02.126563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:02.126840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:02.127111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] PATH_INFO: `/v3/users/363f198e18094e0e84d60f63d3443622` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:02.128274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=363f198e18094e0e84d60f63d3443622)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:02.130218 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:02.174967 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363f198e18094e0e84d60f63d3443622'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.187341 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363f198e18094e0e84d60f63d3443622'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.193939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363f198e18094e0e84d60f63d3443622'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.194606 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363f198e18094e0e84d60f63d3443622'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.195207 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-2d4baa9f-9764-4690-b53c-cfeaf0a59a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363f198e18094e0e84d60f63d3443622'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.208516 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1451/2885] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:26:02 2022] DELETE /identity/v3/users/363f198e18094e0e84d60f63d3443622 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:26:02.382079 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:02.383652 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:02.398244 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3738d6-aac9-423f-b00e-061b2e3e617d admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:02.454449 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:02.455008 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:02.455283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:02.455567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] PATH_INFO: `/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:02.456687 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e81acb21c8b4a439a0ee4ea3ce5aebe)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:02.458635 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:02.468991 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:02.505233 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e81acb21c8b4a439a0ee4ea3ce5aebe', 'request_id': 'req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.512420 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e81acb21c8b4a439a0ee4ea3ce5aebe', 'request_id': 'req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.518735 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e81acb21c8b4a439a0ee4ea3ce5aebe', 'request_id': 'req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.529809 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] The token cache is being invalidate because project 3e81acb21c8b4a439a0ee4ea3ce5aebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:26:02.530120 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.530758 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:26:02.532265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1435/2886] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:26:02 2022] DELETE /identity/v3/projects/3e81acb21c8b4a439a0ee4ea3ce5aebe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:26:12.205135 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:12.526709 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:14.602775 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] REQUEST_METHOD: `POST` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:14.603838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:14.603838 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:14.626649 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.auth.core [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] MFA Rules not processed for user `f56d3e6849ff4268a03ef67c8cf0f813`. Rule list: `[]` (Enabled: `True`). {{(pid=80375) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:26:14.628121 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:14.686050 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1452/2887] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:26:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:26:19.142439 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:19.144529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:19.203712 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ac2-baaf-41c5-90c4-7fa70cb2ed74 admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:19.207203 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:19.207824 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:19.208110 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:19.208382 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:19.210180 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:19.282335 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:19.284402 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:19.296929 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1436/2888] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:26:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:26:22.833421 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:26:22.844306 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60532a-f37f-44b4-ad2a-e64de4566446 None None] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:26:22.848027 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:26:22.848651 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:26:22.848933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:26:22.849211 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:26:22.851427 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:26:22.865174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:26:22.866645 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:26:22.878239 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1453/2889] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:26:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:26:29.291701 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=54 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:26:32.872933 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=31 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:27:35.271206 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:27:35.280770 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855bb5-ffcf-4fc9-9318-f25e3982c280 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:27:35.285314 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:27:35.285979 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:27:35.286262 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:27:35.286538 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:27:35.288536 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:27:35.299009 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:27:35.300444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36c223-17d7-4855-8d8c-b0f62d115101 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:27:35.308680 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1437/2890] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:27:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:27:45.304981 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=6 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:18.229884 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:18.232929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:18.250183 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2006e-f900-40e3-8cac-6e1670be3c0f service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:18.257170 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:18.257809 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:18.258491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:18.258491 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:18.260309 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:18.275545 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:18.277191 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:18.290399 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1454/2891] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:28:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:28:19.403999 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:19.404517 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:19.404631 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:19.427870 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] MFA Rules not processed for user `e0d7c7e659ec440786e599f1c0a64d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:28:19.429103 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:19.488391 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1438/2892] 10.222.0.48 () {58 vars in 967 bytes} [Thu Jul 28 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:28:19.531140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:19.532579 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:19.618326 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d27c5-1273-460c-a6a8-669631ee79b3 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:19.623322 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59f92dd2b31f4a6dbb732ee67575ac1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:19.623948 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:19.624230 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:19.624505 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:19.636386 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:19.638107 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:19.647664 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1455/2893] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Jul 28 09:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 09:28:26.478558 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:26.481605 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:26.554987 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:26.556437 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:26.562487 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33c36-eb1b-4cc7-ae09-8c9aa887f68f None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:26.571116 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:26.571516 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:26.572165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:26.572463 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:26.574637 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:26.591839 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:26.593471 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:26.604749 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1439/2894] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:28:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:28:26.633328 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa5fc8-a789-45f6-acea-e2e4ae43f34a service swift] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:26.638146 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:26.638805 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:26.639085 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:26.639361 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] PATH_INFO: `/v3/users/cafc77c67a844f248b0879d39d5511dd` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:26.640562 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafc77c67a844f248b0879d39d5511dd)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:26.642563 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:26.695643 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafc77c67a844f248b0879d39d5511dd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:26.708412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafc77c67a844f248b0879d39d5511dd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:26.715564 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafc77c67a844f248b0879d39d5511dd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:26.716294 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafc77c67a844f248b0879d39d5511dd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:26.716956 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafc77c67a844f248b0879d39d5511dd'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:26.721249 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:26.736073 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1456/2895] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:28:26 2022] DELETE /identity/v3/users/cafc77c67a844f248b0879d39d5511dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:28:26.929289 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:26.940159 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7315063-7e22-4aaa-9d9a-888f68832f39 service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:27.002190 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:27.002828 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:27.003106 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:27.003377 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] PATH_INFO: `/v3/projects/33b2e26d27c84e738eede31dc3c9ee3a` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:27.005118 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b2e26d27c84e738eede31dc3c9ee3a)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:27.006993 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:27.047966 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:27.058308 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b2e26d27c84e738eede31dc3c9ee3a', 'request_id': 'req-69d29eab-d900-4200-8dd1-93057aec89af'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.065533 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b2e26d27c84e738eede31dc3c9ee3a', 'request_id': 'req-69d29eab-d900-4200-8dd1-93057aec89af'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.072302 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b2e26d27c84e738eede31dc3c9ee3a', 'request_id': 'req-69d29eab-d900-4200-8dd1-93057aec89af'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.083160 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] The token cache is being invalidate because project 33b2e26d27c84e738eede31dc3c9ee3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:28:27.083444 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.084115 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.085815 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1440/2896] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:28:26 2022] DELETE /identity/v3/projects/33b2e26d27c84e738eede31dc3c9ee3a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:28:27.090248 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:27.091686 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:27.104779 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f523b16-6e47-478f-a66d-ca4a946f178e admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:27.162918 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:27.163610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:27.163896 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:27.164169 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] PATH_INFO: `/v3/users/0095212a29754d75bea38b7f8819983f` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:27.165353 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0095212a29754d75bea38b7f8819983f)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:27.167331 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:27.213634 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0095212a29754d75bea38b7f8819983f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.226620 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0095212a29754d75bea38b7f8819983f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.233724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0095212a29754d75bea38b7f8819983f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.234504 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0095212a29754d75bea38b7f8819983f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.235128 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0095212a29754d75bea38b7f8819983f'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.249424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1457/2897] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:28:27 2022] DELETE /identity/v3/users/0095212a29754d75bea38b7f8819983f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:28:27.425179 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:27.426689 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:27.447532 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29eab-d900-4200-8dd1-93057aec89af admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:27.511878 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:27.512572 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:27.512802 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:27.513076 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] PATH_INFO: `/v3/projects/434939b146894b19bbed606c6776b66c` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:27.514283 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] RBAC: Authorizing `identity:delete_project(project_id=434939b146894b19bbed606c6776b66c)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:27.516177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:27.561737 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434939b146894b19bbed606c6776b66c', 'request_id': 'req-97856a52-5ab5-4006-9aa4-98e4d47da00b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.569169 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434939b146894b19bbed606c6776b66c', 'request_id': 'req-97856a52-5ab5-4006-9aa4-98e4d47da00b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.575431 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434939b146894b19bbed606c6776b66c', 'request_id': 'req-97856a52-5ab5-4006-9aa4-98e4d47da00b'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.586528 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] The token cache is being invalidate because project 434939b146894b19bbed606c6776b66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80374) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:28:27.586806 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.587452 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-97856a52-5ab5-4006-9aa4-98e4d47da00b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:28:27.588960 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1441/2898] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:28:27 2022] DELETE /identity/v3/projects/434939b146894b19bbed606c6776b66c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:28:36.733083 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:36.734513 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:36.790484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a3a6f-582a-4274-9536-11e63d85bc7a admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:36.817898 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:36.839815 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40d10d1a51524ba2967695566bb78e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:36.840697 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:36.840974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:36.841243 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:36.843018 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:36.902070 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:36.903446 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:36.913012 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1458/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:28:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:28:37.583472 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:46.902723 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] REQUEST_METHOD: `POST` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:46.903340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:46.903340 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:46.908208 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=34 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:46.922916 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.auth.core [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] MFA Rules not processed for user `f56d3e6849ff4268a03ef67c8cf0f813`. Rule list: `[]` (Enabled: `True`). {{(pid=80374) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 09:28:46.924404 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:46.931165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1442/2900] 10.222.0.48 () {58 vars in 958 bytes} [Thu Jul 28 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 09:28:47.242831 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:47.244265 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:47.305175 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13732b-e035-4533-bbe8-441375f060af service cinder] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:47.308839 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:47.309459 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:47.309746 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:47.310066 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:47.311822 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:47.321763 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:47.323214 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:47.331237 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1459/2901] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:28:47.776026 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:28:47.777660 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:47.858846 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652a0ba-bc10-4d2f-9f36-4bd9b98e990a None None] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:28:47.863856 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:28:47.865099 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:28:47.865398 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:28:47.865674 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:28:47.867514 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:28:47.879126 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:28:47.880771 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:28:47.890622 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1443/2902] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:28:57.327544 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=26 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:28:57.886432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=35 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:29:02.217958 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:29:02.219484 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:29:02.299687 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109120a0-d173-4e7e-b488-d3ed7d4e4d80 service neutron] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:29:02.305223 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96a7013052fd4d48a2463cf51f13861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:29:02.305878 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:29:02.306155 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:29:02.306430 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:29:02.308201 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:29:02.321052 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:29:02.322724 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:29:02.334174 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1460/2903] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:29:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:29:04.693468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:29:04.703567 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4def385-4cb6-4975-a0ad-725325699c92 service placement] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:29:04.707266 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:29:04.707875 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:29:04.708182 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:29:04.708443 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:29:04.710265 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:29:04.772822 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:29:04.774322 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:29:04.784021 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1444/2904] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:29:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:29:12.329140 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=34 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:29:14.779529 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=30 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:29:35.581929 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:29:35.658262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e1a-4647-4539-95e1-4731252ec236 service nova] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:29:35.667866 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:29:35.668916 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:29:35.669412 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:29:35.669939 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:29:35.672368 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:29:35.709355 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:29:35.711082 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:29:35.722004 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1461/2905] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:29:45.717495 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=36 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:31:09.687165 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:09.688627 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:09.745108 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e7c3-8698-4808-a971-6eff3154568a service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:09.748811 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6077e3f936ea42acbc6879758ddc8336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:09.749432 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:09.749717 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:09.750202 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:09.752048 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:09.841197 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:09.842692 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:09.862696 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1445/2906] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 09:31:13.180001 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:13.204997 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7bb5-fa2d-4858-bacd-787bb30144a3 service placement] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:13.223262 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:13.224348 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] REQUEST_METHOD: `GET` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:13.224781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:13.225167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:13.227245 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:13.256840 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:13.263508 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:13.286111 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1462/2907] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:31:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:31:16.564823 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:16.567598 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:16.580177 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2513a8-34aa-4f59-a6d5-65dea43857e0 service neutron] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:16.585597 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:16.586279 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:16.586559 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:16.586838 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] PATH_INFO: `/v3/users/f56d3e6849ff4268a03ef67c8cf0f813` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:16.588001 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56d3e6849ff4268a03ef67c8cf0f813)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:16.590047 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:16.649137 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56d3e6849ff4268a03ef67c8cf0f813'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:16.649913 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56d3e6849ff4268a03ef67c8cf0f813'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:16.663094 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56d3e6849ff4268a03ef67c8cf0f813'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:16.672375 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56d3e6849ff4268a03ef67c8cf0f813'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:16.679854 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56d3e6849ff4268a03ef67c8cf0f813'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:16.681321 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:31:16.686933 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1446/2908] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:31:16 2022] DELETE /identity/v3/users/f56d3e6849ff4268a03ef67c8cf0f813 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:31:16.878607 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:16.892077 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3e0e-2eee-4980-b58e-1589154079d7 service placement] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:16.968982 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:16.969765 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:16.970236 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:16.970610 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] PATH_INFO: `/v3/projects/b688d320376c4492b3ca97b95c5207a5` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:16.971888 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b688d320376c4492b3ca97b95c5207a5)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:16.973894 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:17.028292 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b688d320376c4492b3ca97b95c5207a5', 'request_id': 'req-675e2dfd-469f-4dc6-bd53-de319371d899'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.036007 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b688d320376c4492b3ca97b95c5207a5', 'request_id': 'req-675e2dfd-469f-4dc6-bd53-de319371d899'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.044928 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b688d320376c4492b3ca97b95c5207a5', 'request_id': 'req-675e2dfd-469f-4dc6-bd53-de319371d899'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.058830 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] The token cache is being invalidate because project b688d320376c4492b3ca97b95c5207a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:31:17.059119 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.059792 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.061440 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1463/2909] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:31:16 2022] DELETE /identity/v3/projects/b688d320376c4492b3ca97b95c5207a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:31:17.066086 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:17.068088 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:17.082274 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bf1f2-159f-4d98-8dec-f24143faa8bb admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:17.151850 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:17.152496 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:17.152786 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:17.153066 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] PATH_INFO: `/v3/users/cf2b6a493cbf474bb23d54ea1fea4f74` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:17.154286 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2b6a493cbf474bb23d54ea1fea4f74)` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:17.156387 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:17.205706 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2b6a493cbf474bb23d54ea1fea4f74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.206485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2b6a493cbf474bb23d54ea1fea4f74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.218618 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2b6a493cbf474bb23d54ea1fea4f74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.227547 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2b6a493cbf474bb23d54ea1fea4f74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.236140 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.notifications [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2b6a493cbf474bb23d54ea1fea4f74'} {{(pid=80374) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.242934 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1447/2910] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Jul 28 09:31:17 2022] DELETE /identity/v3/users/cf2b6a493cbf474bb23d54ea1fea4f74 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 09:31:17.435167 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Authenticating user token {{(pid=80375) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:31:17.436781 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.fernet_utils [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80375) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:31:17.451424 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e2dfd-469f-4dc6-bd53-de319371d899 admin admin] Validating token access rules against request {{(pid=80375) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:31:17.522753 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b84f6da05694fb8ade9ceaae63ccb08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66b6795243644c5d8ae6b99e815d8914', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80375) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:31:17.524033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:31:17.524033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:31:17.524033 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] PATH_INFO: `/v3/projects/fe092e0cbbe4425795519f6c4e9c976d` {{(pid=80375) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:31:17.525974 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe092e0cbbe4425795519f6c4e9c976d)` {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:31:17.527035 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] RBAC: Authorization granted {{(pid=80375) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:31:17.548147 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:31:17.579274 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe092e0cbbe4425795519f6c4e9c976d', 'request_id': 'req-d1287352-5932-4aa6-871d-89bf35bf7ea9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.588406 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe092e0cbbe4425795519f6c4e9c976d', 'request_id': 'req-d1287352-5932-4aa6-871d-89bf35bf7ea9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.597016 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe092e0cbbe4425795519f6c4e9c976d', 'request_id': 'req-d1287352-5932-4aa6-871d-89bf35bf7ea9'} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.609362 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] The token cache is being invalidate because project fe092e0cbbe4425795519f6c4e9c976d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80375) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 09:31:17.609590 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.610227 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG keystone.notifications [None req-d1287352-5932-4aa6-871d-89bf35bf7ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80375) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 09:31:17.611767 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: [pid: 80375|app: 0|req: 1464/2911] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Jul 28 09:31:17 2022] DELETE /identity/v3/projects/fe092e0cbbe4425795519f6c4e9c976d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 09:31:27.239448 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:31:27.605776 dv-cinder-851249-5-iscsi devstack@keystone.service[80375]: DEBUG dbcounter [-] [80375] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80375) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 09:34:37.507315 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Authenticating user token {{(pid=80374) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 09:34:37.508779 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:34:37.589212 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9799b-116d-40bb-a3f5-93fb20d94e3e admin admin] Validating token access rules against request {{(pid=80374) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 09:34:37.660280 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9d639d97b415aba4f3772e8ea0f2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65cbf330066d4039af6a9cacf70f36f4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80374) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 09:34:37.660908 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] REQUEST_METHOD: `GET` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 09:34:37.661210 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] SCRIPT_NAME: `/identity` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 09:34:37.661485 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80374) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 09:34:37.663510 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.fernet_utils [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80374) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 09:34:37.675640 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 09:34:37.677400 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081a519-01c4-4d91-9eb2-5abaded55c97 service placement] RBAC: Authorization granted {{(pid=80374) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 09:34:37.687977 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: [pid: 80374|app: 0|req: 1448/2912] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Jul 28 09:34:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 09:34:47.683468 dv-cinder-851249-5-iscsi devstack@keystone.service[80374]: DEBUG dbcounter [-] [80374] Writing DB stats keystone:SELECT=58 {{(pid=80374) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}